Distributed Workflow Orchestration Platform  ·  View 12 of 31  ·  Data

Execution Event Log & State Projection

What the event log is for, and what it is deliberately not used for.

Editable source SVG draw.io All views
Decide
Decide
Append
Append
Project
Project
Serve
Serve
Command path
Command path
Orchestration Service
validates transition
Orchestration Service...
Transactional batch
state + event + outbox
Transactional batch...
Event log
Event log
execution-events
gapless seq per execution
execution-events...
Change feed processor
ordered · at-least-once
Change feed processor...
Projectors
Projectors
Status projector
Status projector
Timeline projector
Timeline projector
Metrics projector
Metrics projector
Read models
Read models
Execution status doc
p95 < 40 ms read
Execution status doc...
SSE status stream
SSE status stream
SQL fact tables
SQL fact tables
Recovery
Recovery
Replay from sequence
operator initiated
Replay from sequence...
Rebuilt projection
swap on catch-up
Rebuilt projection...
one write
one write
append
append
change feed
change feed
rebuild
rebuild
Execution Event Log & State Projection
Execution Event Log & State Projection
Application we own
Application we own
Data store
Data store
Interface / broker
Interface / broker
synchronous
synchronous
event / async
event / async
batch
batch
This is event sourcing used for recovery and audit, not for reads. Current state is materialised in the same write as the event, so no read path depends on replaying the log — but every projection can be thrown away and rebuilt.
This is event sourcing used for recovery and audit, not for reads. Current state is materialised in the same write as the event, so no read path depends on replaying the log — but every projection can be thrown away and rebuilt.
v 1.0 · owner Data & AI Global Practice · date 2026-08
v 1.0 · owner Data & AI Global Practice · date 2026-08
Text is not SVG - cannot display

Decisions

  • This is event sourcing for recovery and audit, not for reads. Current state is materialised in the same write as the event, so no read path replays anything and no query is at the mercy of projection lag.
  • Full event sourcing — where state exists only as a fold over events — was rejected for V1. It would put replay latency on the critical path of every orchestration decision to solve a problem the platform does not have.
  • Every projection is disposable by design. If a projector has a bug, the fix is to rebuild from sequence zero and swap, which is only possible because the log is authoritative and gapless.

What this buys

  • The complete answer to 'what happened to execution 123', reconstructed from the log rather than from logs that may have been sampled or aged out.
  • Operator replay (view 20) and projection rebuild are the same mechanism with different starting points, so the recovery path is exercised by ordinary operations rather than only in incidents.
  • Nine event types cover the lifecycle: WorkflowStarted, TaskScheduled, TaskStarted, TaskCompleted, TaskFailed, TaskRetried, TaskDeadLettered, WorkflowCompleted, WorkflowFailed.

Risks

  • The change feed guarantees at-least-once delivery to projectors, so every projector must be idempotent on sequence_no. A projector that increments a counter without checking is a silent data bug.
  • 90-day event retention means executions older than that cannot be fully reconstructed after archive. The archive is queryable but not fast, and the boundary should be agreed with audit.
  • Projection lag under peak load is unmeasured until load testing. The SSE status stream is the consumer most sensitive to it.