Distributed Workflow Orchestration Platform  ·  View 10 of 31  ·  Data

Execution & Metadata Data Model

The thirteen entities an orchestration decision reads or writes.

Editable source SVG draw.io All views
tenant
tenant_id PK
name
tier (bronze|silver|gold)
max_concurrent_tasks
max_concurrent_executions
priority_class_default
tenant...
workflow_definition
workflow_id PK
tenant_id FK -> tenant
name
latest_version
owner_team
workflow_definition...
workflow_version
workflow_version_id PK
workflow_id FK -> workflow_definition
version (immutable)
content_hash
dag_json
retry_policy_json
published_at
status (active|deprecated)
workflow_version...
schedule
schedule_id PK
tenant_id FK -> tenant
workflow_id FK -> workflow_definition
cron_expression
timezone
next_fire_at
misfire_policy
schedule...
trigger_subscription
subscription_id PK
workflow_id FK -> workflow_definition
event_type
filter_expression
input_mapping
trigger_subscription...
workflow_execution
workflow_execution_id PK
tenant_id FK -> tenant
workflow_version_id FK -> workflow_version
status
correlation_id
idempotency_key
input_ref
output_ref
created_at / started_at / completed_at
_etag (optimistic concurrency)
workflow_execution...
task_execution
task_execution_id PK
workflow_execution_id FK -> workflow_execution
task_id (DAG node)
task_type
status
attempt_count
depends_on[]
priority_class
next_visible_at
_etag
task_execution...
task_attempt
task_attempt_id PK
task_execution_id FK -> task_execution
attempt_no
worker_id
started_at / ended_at
outcome
error_code
error_message
trace_id / span_id
task_attempt...
execution_event
event_id PK
workflow_execution_id FK -> workflow_execution
sequence_no (gapless)
event_type
payload_json
occurred_at
actor
execution_event...
outbox_message
outbox_id PK
workflow_execution_id FK -> workflow_execution
destination_queue
body_ref
created_at
published_at (null = pending)
outbox_message...
dead_letter_record
dead_letter_id PK
task_execution_id FK -> task_execution
tenant_id FK -> tenant
attempt_count
error_code / error_message
stack_trace
original_payload_ref
disposition (open|retried|discarded)
dead_letter_record...
idempotency_record
task_execution_id PK
handler_name
side_effect_ref
result_ref
completed_at
ttl_seconds
idempotency_record...
concurrency_lease
lease_key PK (scope:tenant:type)
holder_instance_id
fencing_token (monotonic)
acquired_at
expires_at
concurrency_lease...
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : N
1 : 0..1
1 : 0..1
1 : 0..1
1 : 0..1
Execution & Metadata Data Model
Execution & Metadata Data Model
workflow_execution and task_execution share the /workflowExecutionId logical partition, which is what lets a state change and its outbox message be written in one transactional batch.
workflow_execution and task_execution share the /workflowExecutionId logical partition, which is what lets a state change and its outbox message be written in one transactional batch.
v 1.0 · owner Data & AI Global Practice · date 2026-08
v 1.0 · owner Data & AI Global Practice · date 2026-08
Text is not SVG - cannot display

Modelling decisions

  • task_attempt is a separate entity from task_execution rather than a counter on it. Retry analysis, error taxonomy and the 'why did it take 47 seconds' question in view 25 all need per-attempt timing, worker identity and trace id.
  • outbox_message is part of the model, not an implementation detail. Its published_at being null is the entire definition of 'a task that must be dispatched', and the change feed over it is the dispatcher.
  • concurrency_lease carries a fencing token even though it is a fairness mechanism rather than a correctness one. Uniformity with the scheduler lease in view 30 is worth more than the bytes saved.

Keys and consistency

  • workflow_execution and task_execution share the /workflowExecutionId logical partition. That co-location is what allows a state change, its event and its outbox row to commit as one transactional batch.
  • Both carry _etag and every transition is an ETag-guarded replace. Two orchestrators racing the same completion means one succeeds and one retries against fresh state, which is correct rather than merely unlikely.
  • execution_event.sequence_no is gapless per execution, assigned inside the same batch. A gap is therefore evidence of a bug, which makes it a usable invariant to alert on.

Assumptions

  • Rendered as a relational model for reviewability; the physical store is Cosmos, where several of these entities are documents in one container distinguished by a type field.
  • idempotency_record is written by the worker, not the orchestrator, and expires at 7 days on the assumption that no retry window exceeds it.
  • The reporting star schema in Azure SQL is derived from these entities and is not modelled here.