Distributed Workflow Orchestration Platform  ·  View 21 of 31  ·  Runtime

Backpressure & Concurrency Governance

Five control scopes against four enforcement points, and the one place a request is actually refused.

Editable source SVG draw.io All views
API admission
API admission
Dispatch decision
Dispatch decision
Worker runtime
Worker runtime
Infrastructure
Infrastructure
Tenant
Tenant
Submission quota
APIM · 600/min gold
Submission quota...
Concurrency lease
Redis · 500 / 100 / 20
Concurrency lease...
Per-tenant prefetch cap
Per-tenant prefetch cap
Dedicated namespace
tier-1 escape hatch
Dedicated namespace...
Task type
Task type
Per-type lease
identity-check ≤ 50
Per-type lease...
Handler semaphore
in-process
Handler semaphore...
Deployment per type
independent scaling
Deployment per type...
Downstream
Downstream
Circuit state check
skip dispatch when open
Circuit state check...
Adaptive rate limiter
honours Retry-After
Adaptive rate limiter...
LLM token budget
per tenant per hour
LLM token budget...
Queue pressure
Queue pressure
429 above depth
> 200K messages
429 above depth...
Priority class routing
high · standard · bulk
Priority class routing...
Max concurrent calls
8 · 32 · 64
Max concurrent calls...
KEDA bounds
min/max replicas
KEDA bounds...
Cluster
Cluster
Pod resource limits
CPU · memory · GOMAXPROCS
Pod resource limits...
Node pool maxima
cluster autoscaler ceiling
Node pool maxima...
Backpressure — Control Scope Against Enforcement Point
Backpressure — Control Scope Against Enforcement Point
Interface / broker
Interface / broker
Data store
Data store
Application we own
Application we own
Queue / topic
Queue / topic
Security / platform
Security / platform
Decision point
Decision point
Risk / gap
Risk / gap
Admission is the only place a request is refused. Everywhere after it, pressure is expressed as delay rather than rejection, because a task already accepted has been promised at-least-once execution.
Admission is the only place a request is refused. Everywhere after it, pressure is expressed as delay rather than rejection, because a task already accepted has been promised at-least-once execution.
v 1.0 · owner Data & AI Global Practice · date 2026-08
v 1.0 · owner Data & AI Global Practice · date 2026-08
Text is not SVG - cannot display

The governing rule

  • Admission is the only place work is refused. Everywhere after it, pressure is expressed as delay rather than rejection, because a task that has been accepted has been promised at-least-once execution and dropping it would break that promise.
  • 100,000 queued tasks and 10 workers is not a scaling problem, it is a fairness problem. Concurrency leases per tenant and per task type mean the tenth tenant is not starved by the first, regardless of queue order.
  • Limits are configuration, not code. Tenant tier drives the numbers — gold 500, silver 100, bronze 20 concurrent tasks — so a commercial change is a policy change rather than a deployment.

Why four enforcement points

  • Each catches what the others cannot. API Management limits submission rate but knows nothing about how long tasks run; the dispatch lease limits concurrency but cannot protect a saturated pod; the worker semaphore protects the pod but not the downstream; KEDA bounds the fleet but not fairness within it.
  • Priority is expressed as queue selection at dispatch, which is the only place the tenant's tier and the task's class are both known.
  • The downstream ceiling is the real limit. The LLM gateway's token budget and the identity SaaS's 50 rps are enforced by the circuit breaker rather than by hope, and view 24 stops scaling into them.

Risks

  • Redis-held leases are soft state. On Redis failure they fail open at a globally reduced concurrency cap, which preserves liveness at the cost of exact fairness — an explicit trade, stated in view 31.
  • A tenant permanently at its limit looks identical to a stuck platform unless WAITING_SLOT is surfaced per tenant. That dashboard is a launch requirement, not a nice-to-have.
  • Queue-depth-based admission rejection at 200,000 messages is a blunt last resort. It protects the platform and will annoy a tenant who is within their own quota, so it is alarmed as an incident when it fires.