intermediate 2 min answer

For each of these, choose a queue or a stream and justify it — order fulfilment tasks, an audit trail, cache invalidation, and rebuilding a search index.

queuesstreamsretentionreplay
Show the full answer Hide the answer

Order fulfilment tasks — queue

Each task is consumed once by one worker and is then irrelevant. Nothing re-reads it; no second consumer needs the same task.

What a queue gives that matters here: per-message acknowledgement (one poison message does not block everything behind it, which it would in a partitioned stream), visibility timeouts, and built-in dead lettering. Competing consumers scale throughput naturally.

Using a stream here means paying for partitions, consumer groups, offset management and rebalancing to obtain replay you never use.

Audit trail — stream

Ordered, retained, immutable, and read by consumers you have not thought of yet — compliance reporting, security analytics, a future investigation. Retention is the requirement.

Log compaction is wrong here; you want the full history, not the latest value per key. Retention should be set by the regulatory period, and the storage cost budgeted deliberately.

Cache invalidation — stream, and compacted

Every cache node needs every message, so this is broadcast rather than work distribution — a queue's consume-once semantics are simply wrong.

Compaction matters: a node that restarts or joins can replay the compacted topic to obtain the current state of every key rather than starting cold. That converts cache warm-up from a slow database-driven process into a fast sequential read.

Rebuilding a search index — stream

This is the case that only a stream supports. Rebuilding means reading history from the beginning, which requires retention plus independent offsets. Reset the consumer group's offsets and reprocess.

It also demonstrates the strongest argument for streams generally: a new consumer can be added later and bootstrap itself from history without the producer knowing or changing.

The decision rule

Do you need to re-read, or will another consumer need this later? If yes, stream. If each message is handled once and then irrelevant, queue — and take the simpler operational model.

What a strong answer adds

Noting that the two compose, and that many estates run both deliberately: a stream as the durable system of record, with queues fed from it for task distribution where per-message acknowledgement and dead lettering matter. That gets replay where it is valuable and per-message semantics where they are.