Messaging & Queues
Decoupling producer from consumer, and the semantics that come with it.
4 to work through
-
intermediate
A notification service must deliver push notifications for mentions. What delivery guarantee do you offer, and how do you avoid sending duplicates?
2 min answer -
intermediate
A team proposes an event-streaming platform for a background job workflow processing a few hundred thousand jobs a day. What is gained, what is paid for, and when is a database-backed queue the better engineering decision?
2 min answer -
intermediate
For each of these, choose a queue or a stream and justify it — order fulfilment tasks, an audit trail, cache invalidation, and rebuilding a search index.
2 min answer -
advanced
A flash sale causes an incident. Two hours later the fault is fixed but 4 million jobs are backed up and notifications are hours late. Walk through recovery and prevention.
3 min answer
3 terms in this topic
Fan-out on Write vs Fan-out on Read
Whether an event is copied to every recipient's store at publish time, or assembled from sources at read time - and why large systems need both.
conceptMessage Ordering
The guarantee about the sequence in which messages are delivered — normally per-partition or per-group only, and lost the moment consumption is paral…
patternMessage Queue
A durable buffer between a producer and a worker that decouples them in time, absorbs bursts, and turns a synchronous dependency into a retryable one.
Neighbouring topics
Distributed Systems
General material on partial failure, coordination and distributed reasoning.
CAP & PACELC
What you must give up during a partition, and the latency choice the rest of the time.
Consistency Models
Linearizable, sequential, causal, eventual, and the session guarantees between them.
Idempotency
Making an operation safe to repeat, because a client that times out cannot know.
Retries & Backoff
Exponential backoff, jitter, retry budgets, and how retries become the outage.
Timeouts & Deadlines
Per-hop timeouts that do not compose, and the deadline budget that replaces them.
Circuit Breakers
Failing fast on a broken dependency, and what you fail fast to.
Backpressure & Flow Control
Telling callers to slow down instead of buffering into congestion collapse.
Load Shedding
Rejecting some work deliberately so the rest can be served correctly.
Bulkheads & Isolation
Partitioning resources so one dependency cannot starve the others.
Leader Election
Agreeing who is in charge, and fencing the one who no longer is.
Consensus Protocols
Raft, Paxos and quorums — what they guarantee and what they cost.
Distributed Locking
Mutual exclusion across machines, and why it is harder than it looks.
Distributed Transactions
Two-phase commit, its blocking failure mode, and when it is still reasonable.
Sagas & Compensation
Replacing atomicity with semantic undo, and ordering the irreversible steps last.
Service Discovery
Finding a healthy address for something whose instances are ephemeral.
Event Streaming
Retained ordered logs, consumer offsets, partitions and replay.
Clocks & Ordering
Why wall clocks lie, and how logical clocks and versions restore order.
Failure Modes
Slow rather than down, partial, grey, and failing while reporting success.