Boundary Traffic Cost

Every boundary between components has a price that is paid per interaction, not per deployment: serialisation, network transfer, a coordination record, a retry policy, a failure mode, and latency. A decomposition that looks elegant on a diagram can be ruinous in production if the interaction rate is high and the payloads are large.

Boundary traffic cost is what crosses the boundary, multiplied by how often it crosses. It is calculable from a design document before any code exists, and it is very rarely calculated.

Why it matters

Decomposition is usually argued on qualitative grounds — separation of concerns, single responsibility, independent deployability. None of those arguments contain a number, and the cost of the boundary is a number. Teams therefore commit to a structure whose recurring cost they have never estimated, and discover it only when the bill arrives or the latency budget is exhausted.

The failure is especially sharp when the boundary sits inside a loop. A boundary crossed once per user request is a design choice; a boundary crossed once per video frame, per row, or per event is a multiplier, and the multiplier is set by the workload rather than by the traffic.

Implementation patterns

• Estimate before committing: bytes per crossing × crossings per unit of work × units of work per second. Do this in the design review, where changing the answer is free.
• Put the boundary outside the loop. If steps must exchange large state repeatedly, the boundary belongs around the whole loop, not between its iterations — pass a job, not a frame.
• Batch across the boundary where the boundary must stay: amortise coordination over many items.
• Pass references rather than payloads when the consumer may not need the whole object, accepting the extra fetch as the cheaper of the two.
• Keep the module boundary without the network boundary. In-process modules with enforced interfaces get most of the design benefit at none of the traffic cost, and remain splittable later.
• Watch the coordination bill separately from the compute bill, because orchestrators priced per state transition make the coordination cost scale with the unit of work, which is often far finer than the request.

Industry example

Prime Video's video-quality monitoring pipeline decomposed frame-level analysis into functions per step, coordinated by a workflow engine, with intermediate frames passed between steps through object storage. Two costs dominated and neither was compute: state transitions billed per frame rather than per stream, and large intermediate payloads crossing a network boundary at every step. Consolidating the steps into one long-running process moved the frames into memory and removed the orchestration, reportedly cutting cost by around 90% for that component.

The generalisable part is narrow and worth stating precisely: function-per-step decomposition is wrong when the steps are chatty and the payloads are large. It is not evidence that serverless does not scale, nor a verdict on microservices — Amazon runs an enormous number of services, and one team reversing one internal decomposition says nothing about the style.

Failure scenarios

• A boundary inside a per-item loop, where cost scales with items rather than with requests.
• Intermediate state routed through object storage because it is convenient, making every step pay a network round trip for data the next step immediately discards.
• Orchestration priced per transition applied to a fine-grained unit of work.
• Latency budget consumed by hops — five sequential services each adding a few milliseconds of transport and serialisation before doing any work.
• Retry amplification, where each boundary adds its own retry policy and a slow dependency is hit multiplicatively.
• Cost discovered in the invoice rather than in the design, by which point the structure is entrenched.

Trade-offs

Boundaries buy real things: independent deployment, fault isolation, independent scaling, and clear ownership. Minimising boundary traffic cost by merging everything sacrifices all of them, and a system with no boundaries has a different and equally expensive set of problems.

The trade is not "fewer boundaries" but "boundaries placed where the traffic across them is low relative to the isolation they buy." A boundary between two components that exchange a small message once per user request is nearly free and worth having. A boundary between two components that exchange megabytes thousands of times per second is a design error regardless of how clean it looks.

Interview question

"Here is a pipeline design with six steps as separate services. Estimate what it costs per request before we build it — tell me what you need to know, and tell me which of the six boundaries you would remove first and why."