Review this platform for a food-delivery business of Zomato's kind - self-managed Kafka on 9 VMs, self-managed Postgres with a hand-written failover script, self-managed Redis, self-managed Elasticsearch, all on cloud virtual machines, run by a 6-person platform team that is also on call for the consumer app. What would you hand to a managed service, what would you keep, and what would you leave alone?
Show the full answer Hide the answer
What is actually required
Four stateful systems, each with its own upgrade path, failure modes, backup and restore procedure, capacity model and version-specific defects. The constraint is not money and not skill - it is six people's attention, shared with being on call for the product. Any review that does not start there will produce a list of technically defensible choices that collectively cannot be operated.
What I would hand over first
Postgres, specifically because of the hand-written failover script. That script is the highest risk object in the estate: it runs rarely, is therefore never well tested, and executes under the worst conditions imaginable. A managed database's failover runs continuously across the provider's whole fleet, which is the only way failover automation ever becomes trustworthy. This is the one hand-back that reduces the probability of a severe incident rather than merely saving time.
Redis second. The operational value a small team adds over a managed cache is close to zero, and the failure modes - eviction policy, persistence configuration, failover behaviour - are well covered by the managed offering.
Elasticsearch third, with a caveat: version and plugin dependencies are the usual blocker, so check them before promising the migration.
What I would keep self-managed
Kafka, conditionally. Two tests, both arithmetic:
- Cost at your volume. Managed streaming at high sustained throughput can cost several times self-managed - but compare it against the fully loaded cost of the engineering time, not against the virtual machine bill. Two engineers' attention is roughly the price of a lot of managed throughput.
- A genuine version or feature dependency. If something in the stack needs a configuration the managed service does not expose, that is a real reason and it should be written down so it can be revisited.
If neither test gives a clear answer, hand it over - the default should favour fewer systems owned, because the burden compounds.
What I would leave alone even though it looks odd
The 9 Kafka virtual machines, if that is over-provisioned. Spare capacity in a system you keep is cheap insurance, and resizing a cluster is exactly the kind of risky, low-value work a six-person team should not be doing this quarter.
What it costs
- A higher invoice, and the saving is in incidents that do not happen - which is impossible to put on the same slide.
- Upgrade windows you do not control. This is the genuine downside: managed services force minor-version upgrades on a schedule, so the application has to tolerate a failover at a time the provider picks. Design for it, and test it deliberately, rather than discovering it.
- Less tuning freedom, which matters for a handful of workloads and almost never for the ones teams claim it does.
How I would argue this in the review
Not on cost per unit, which favours self-managed and is the wrong frame. On the number of systems six people can be good at, and on which failures would be severe. "We own Kafka because our volume makes it worth a person, and we own nothing else" is a defensible platform strategy. "We own four stateful systems because each one was cheaper that way" is four sets of incident reviews waiting to be written.
When this is the right architecture
A team of 25 with a dedicated database group, or a workload whose cost curve genuinely breaks the managed pricing model, can run all four and run them well. The trigger that should prompt re-opening the decision is a headcount change in either direction - and that is worth writing into the decision record, because the choice follows the team size and not the technology.