A company split its monolith into 14 services. Deployments now require a coordinated release plan across teams and one user action traverses six synchronous hops. What went wrong and what would you do?
Show the full answer Hide the answer
What is being tested
Whether you can diagnose a distributed monolith, and whether you understand that the diagnosis is about coupling and cohesion, not about the number of services.
The diagnosis
The services were split on the wrong axis. Two symptoms give it away.
Coordinated releases mean the boundaries do not contain change. A feature requires modifying several services because the behaviour it changes is spread across them — a cohesion failure. The usual cause is splitting by technical layer or by database entity ("Customer Service", "Order Service", "Product Service") rather than by capability, so no single service can complete a meaningful business action alone.
Six synchronous hops mean the boundaries do not contain the request. Each hop multiplies latency, multiplies failure probability, and creates temporal coupling — if any one of the six is down or slow, the user action fails. Availability is now the product of six numbers instead of one, and the p99 is the sum of six p99s, which is much worse than most people expect.
The organisation has bought every cost of distribution — network failures, distributed debugging, deployment orchestration, data consistency across boundaries — and none of the benefit, because nothing can actually be deployed or scaled independently.
What to do, in order
- Measure before moving. Which services always change together? Commit history over the last six months answers this better than any opinion. Which hops are on the critical user path? Distributed tracing answers that.
- Merge, do not split further. Services that always deploy together should become one service. This is the counterintuitive step and usually the highest value. Reducing 14 services to 5 that align with real capabilities removes most of the pain.
- Break the synchronous chains that remain. For each hop, ask whether the caller genuinely needs the answer to respond to the user. If not, make it an event. If yes, consider whether the data can be replicated locally instead of fetched.
- Redraw the survivors around capabilities. A capability boundary can complete a business action end to end and owns its data.
- Only then invest in the platform. If distribution is genuinely warranted for what remains, the tracing, service discovery and standardised resilience libraries are the price of entry — but buying them for a system that should be consolidated is spending money to preserve the problem.
The uncomfortable conclusion
Sometimes the right answer is to go back to a modular monolith with enforced internal boundaries. That is not a failure or an admission of defeat; it is recognising that the boundaries were the valuable part and the network was the cost. Uber's later consolidation of a very large service estate into coarser domain-oriented groupings is a public example of an organisation re-pricing exactly this trade.
Common weak answers
Adding a service mesh, which makes the six hops observable and retryable without making them fewer. Adding a caching layer to hide the latency, which adds staleness to a correctness problem. Blaming the teams for poor discipline rather than the boundaries for being in the wrong place.