Term Kind Topic What it is
Backfill practice Batch Orchestration Reprocessing historical periods through a pipeline after fixing a defect or adding a field, at a scale the pipeline was not sized for.
Batch Orchestration Workflow Orchestration, DAG Scheduling concept Batch Orchestration Running interdependent data jobs in the right order with retries, backfills and failure handling, expressed as a dependency graph rather than a schedule.
DAG Dependency concept Batch Orchestration The declared edge stating that one task must not start until another has succeeded, and the difference between that and merely running later.
Logical Date Partitioning Execution Date, Idempotent Reprocessing practice Batch Orchestration Making every batch task a pure function of a logical date, so that any run can be repeated safely and a backfill is a range of independent executions.
Wall-Clock Schedule Hazard Local-Time Trigger Hazard, DST Schedule Ambiguity concept Batch Orchestration The class of batch faults caused by triggering work on a civil clock that repeats, skips or shifts, producing duplicate or missing runs that no job reports as an error.
Zombie Task Orphaned Task, Heartbeat-Expired Task concept Batch Orchestration A task whose worker has stopped reporting to the scheduler but whose process may still be running and writing, so the orchestrator retries work that has not actually stopped.