A workflow succeeds in one service, then the next service crashes before recording the event. How do transactional outbox, durable queues, retries, idempotent consumers and reconciliation each contribute, and which of them is not optional?

The gap being closed

The problem is that a state change and the publication of that change are two operations, and any failure between them loses work or invents it. No amount of retrying fixes it because the retry has nothing to read.

What each mechanism contributes

• Transactional outbox — not optional. The state change and an outbox row commit in one local transaction. A separate relay publishes the row afterwards. This converts "did the write happen and the message get sent" into a single atomic question with a durable answer. Everything else is downstream of getting this right.
• Durable queue. Guarantees the message survives broker restarts and consumer crashes. Necessary but insufficient — it protects the message once published, not the gap before publication.
• Retries. Handle the transient failure. They are the reason the outbox relay can be simple and the reason duplicates exist.
• Idempotent consumers — not optional. Because the relay may publish twice (it crashed after publishing, before marking the row sent), at-least-once is what you actually have. A consumer that is not idempotent turns a correct delivery guarantee into a correctness bug.
• Reconciliation. The backstop for everything the mechanisms above miss: a poisoned message, a bug in the consumer, a manual intervention. It is the only control that finds problems the pipeline does not know it has.

Where a durable workflow engine changes the picture

A workflow engine such as Temporal moves this machinery out of application code. The workflow's state is persisted by the engine, activities are retried according to declared policy, and a worker crash resumes from the last completed step rather than from the beginning. The value is not that it makes failures impossible — it makes the recovery code someone else's problem, and application-level orchestration of long-running processes is where hand-written implementations fail most reliably.

The trade is a new operational dependency, a new programming model with real constraints (determinism in workflow code, versioning of running workflows), and a new place for outages. It earns that cost when workflows are long-running, numerous, and expensive to get wrong.

The judgement

Exactly-once delivery does not exist across a network. Exactly-once effect does, and it is built from at-least-once delivery plus idempotent consumers. Teams that pursue the first spend months and end up building the second anyway, with less understanding of where the duplicates were absorbed.