Design the order-fulfilment saga for a marketplace where sellers can cancel, buyers can cancel, payments can fail asynchronously hours later, and items are unique. What states and compensations are required, and where does orchestration beat choreography?

Why this saga is harder than the textbook one

Three properties break the tidy version:

• Compensation may arrive long after completion. A payment can fail or be reversed hours later, well after the item shipped. The saga cannot assume it ends when the happy path ends.
• Cancellation can come from either party at almost any state, including concurrently with the system's own transitions.
• Items are unique, so a released reservation may be immediately taken by someone else — meaning compensation is not always reversible in the way the model assumes.

The state model

States are business states, not technical ones:

reserved → payment_pending → paid → fulfilment_requested → shipped → completed
                    ↓            ↓             ↓               ↓
              reservation_   refunded/    cancelled_       return_
               released      cancelled    pre_dispatch     initiated

Two rules make this tractable:

1. Every state names who may leave it and how. "Buyer may cancel until shipped; after shipped the path is return_initiated, which is a different process with different economics."
2. Compensations are forward transitions, not undos. There is no edge back to reserved. A refund moves to refunded, a distinct terminal state with its own accounting consequences.

Where orchestration beats choreography

Choreography — each service reacting to events with no central coordinator — is excellent for simple flows and terrible here, for a specific reason: no single component knows the state of the order.

• Answering "why is this order stuck?" requires reconstructing state from the event logs of five services.
• Timeouts have no owner. If fulfilment never responds, which service is responsible for noticing?
• Concurrent cancellation and progression race, and with no central arbiter both can succeed.

An orchestrator — a durable workflow that owns the state machine, issues commands and awaits replies — buys exactly what is missing: an authoritative current state, a natural home for timeouts, a single place to enforce that only one transition applies at a time, and an answerable question during an incident.

The cost is real: a central component every order flows through, a new operational dependency, and the risk that the orchestrator becomes a distributed monolith accumulating business logic that belongs in services. Keep it thin — sequencing and timeouts, not pricing rules.

The rule of thumb: choreograph events that are notifications ("order shipped" for analytics, search, email). Orchestrate transitions that carry obligations ("charge this", "reserve that", "refund if not shipped within N days").

The details that decide whether it works in production

• Idempotency keys on every command, because at-least-once delivery means every step reruns.
• Explicit timeouts on every wait state, with a defined action. A saga with a state that has no timeout is a saga with orders that stay stuck forever.
• Compensation is best-effort and must be observable. A refund can itself fail; the saga needs a state for "compensation failed, human required" rather than retrying forever in silence.
• A reconciliation job comparing saga states against payment provider and fulfilment records, because in a system this asynchronous, drift is not an anomaly — it is a certainty, and the only question is whether you detect it or a customer does.