advanced 2 min answer

A streaming platform's cloud bill is dominated by a line item nobody recognises: inter-zone data transfer. Explain and fix.

finopsstreamingnetworking
Show the full answer Hide the answer

What the interviewer is testing

Whether you know the specific and frequently overlooked network economics of a replicated log in the cloud.

The explanation

Two multipliers, both continuous:

Replication. With a replication factor of three, every produced byte is written to a leader and then transferred to two followers. If those brokers are in different availability zones — which they must be for zone fault tolerance — each transfer crosses a billing boundary.

Consumption. By default a consumer reads from the partition leader, wherever it is. With consumers spread across three zones and leaders distributed across three zones, roughly two-thirds of all consumption traffic crosses zones. With multiple consumer groups, that multiplies again — ten consumer groups means ten times the cross-zone read traffic for the same data.

For a high-throughput topic this can exceed the compute cost of the entire platform, and it appears as a network line item with no obvious owner.

The fixes

Rack-aware or zone-aware consumption, where consumers read from a replica in their own zone. This is supported by modern brokers and typically eliminates the majority of the consumption charge. It is the single highest-impact change.

Zone-aware producer and consumer placement, keeping producers close to leaders where the workload allows.

Compression on the producer, which reduces every byte transferred — replication and consumption alike — and is often left at none.

Consolidate consumer groups. Ten services each consuming the same topic independently is ten copies of the traffic; a fan-out service reading once and distributing may be cheaper, though it adds a component.

Tiered storage so long retention lands on object storage rather than broker disk.

What a strong answer adds

Questioning the topology: three-zone replication is the right default for fault tolerance and it is a deliberate purchase. Some non-critical topics may reasonably run with lower replication or within a zone, and making that a per-topic decision rather than a platform default can be a large saving.

Common weak answers

Reducing the replication factor across the board, which trades durability without analysis. Assuming the bill is compute.