Distributed Workflow Orchestration Platform  ·  View 24 of 31  ·  Operations

Queue-Driven Autoscaling with KEDA

What the fleet scales on, why it is not queue depth, and where scaling stops helping.

Editable source SVG draw.io All views
Signal
Signal
Active message count
per task queue
Active message count...
Oldest message age
the SLO that matters
Oldest message age...
Decide
Decide
KEDA ScaledObject
azure-servicebus trigger
KEDA ScaledObject...
queueLength 20 / replica
cooldown 300 s
queueLength 20 / replica...
HPA behaviour
up fast · down slow
HPA behaviour...
Scale out
Scale out
Worker replicas 2 → 60
per deployment bounds
Worker replicas 2 → 60...
Cluster autoscaler
spot first, on-demand fallback
Cluster autoscaler...
Absorb
Absorb
900 tasks/s sustained
peak drain target
900 tasks/s sustained...
Tenant leases still cap
fairness survives scale-out
Tenant leases still cap...
Downstream ceiling
400 rps · scaling cannot pass it
Downstream ceiling...
Scale in
Scale in
Cooldown then scale in
5 min stable
Cooldown then scale in...
worker-rag → 0
scale-to-zero, cold start 25 s
worker-rag → 0...
PodDisruptionBudget
min 2 during drain
PodDisruptionBudget...
circuit opens · stop scaling
circuit opens · stop scaling
primary trigger
primary trigger
Queue-Driven Autoscaling with KEDA
Queue-Driven Autoscaling with KEDA
Queue / topic
Queue / topic
Security / platform
Security / platform
Application we own
Application we own
Data store
Data store
Risk / gap
Risk / gap
failure / alternate
failure / alternate
synchronous
synchronous
Depth alone is the wrong trigger: 200,000 cheap messages and 200 expensive ones look identical. Oldest-message-age is scaled against the task scheduling SLO of p95 under one second, and depth is the secondary signal.
Depth alone is the wrong trigger: 200,000 cheap messages and 200 expensive ones look identical. Oldest-message-age is scaled against the task scheduling SLO of p95 under one second, and depth is the secondary signal.
v 1.0 · owner Data & AI Global Practice · date 2026-08
v 1.0 · owner Data & AI Global Practice · date 2026-08
Text is not SVG - cannot display

Decisions

  • Oldest-message-age is the primary trigger and depth is secondary. Depth alone is the wrong signal: 200,000 cheap messages and 200 expensive ones look identical, and only one of them is late.
  • The trigger is scaled directly against the SLO — p95 task scheduling under one second — so the autoscaler is defending a stated commitment rather than a comfortable-looking queue length.
  • Scale up fast and scale in slow, with a 5-minute cooldown. Thrashing a worker fleet costs more in cold starts and abandoned leases than the compute it saves.

Numbers

  • queueLength 20 per replica, worker-http bounded 2 to 60, worker-customer360 2 to 30, worker-rag 0 to 20 with scale-to-zero and a 25-second cold start, worker-notify 2 to 15.
  • Peak drain target 900 tasks per second sustained. Cluster autoscaler adds spot nodes first and falls back to on-demand when spot capacity is unavailable.
  • PodDisruptionBudget keeps a minimum of 2 replicas during node drain, so a scale-in and a node upgrade cannot combine to empty a queue's consumers.

Risks

  • Scaling into a downstream ceiling is the most likely way this platform causes an outage somewhere else. When the circuit opens, KEDA stops scaling that deployment — autoscaling is subordinated to the circuit breaker, not the other way round.
  • Per-tenant concurrency leases still cap throughput after scale-out, so adding pods for a single saturated tenant achieves nothing. Fairness deliberately survives elasticity.
  • Scale-to-zero on worker-rag trades 25 seconds of cold start for idle cost. It is correct for a batch-shaped workload and wrong for an interactive one, so it is per-deployment configuration.