A team wants to self-manage Kafka on Kubernetes to save money against the managed service. Evaluate.

The comparison they have probably made

Instance cost versus managed service list price. That comparison usually favours self-managing by a wide margin, and it omits most of the actual cost.

What the comparison must include

Engineering time to operate, continuously: broker replacement, partition rebalancing, storage expansion, version upgrades (Kafka upgrades are not trivial and involve rolling restarts with careful ordering), certificate rotation, and capacity planning.

On-call burden. Someone must be able to diagnose an under-replicated partition, a stuck controller or a consumer group rebalancing loop at 3 a.m. That is specialist knowledge, it must exist in more than one head, and it must be maintained.

The cost of getting it wrong. Kafka's failure modes are recoverable by people who know it and data-losing by people who do not. min.insync.replicas, unclean.leader.election, and retention configured wrongly are each capable of silent data loss.

Opportunity cost. The engineers operating it are not building product.

And the cost that is genuinely on their side: at very large scale the managed premium is substantial, and networking costs in the managed service can be significant.

The questions I would ask

What is the actual throughput and retention? The economics differ by an order of magnitude between a few thousand messages a second and a few million. Below a modest threshold, the managed premium is small in absolute terms and the answer is easy.

Do we have the expertise, in more than one person? A single expert is a resignation away from an unsupportable system.

Is this our core domain? Almost certainly not, which is the strongest argument.

Have we considered the middle option? Managed Kafka from a specialist vendor, or a managed Kafka-compatible service, often sits between the two on both cost and control.

Do we need Kafka at all? A meaningful share of Kafka deployments would be adequately served by a managed queue plus a managed stream, at a fraction of the operational cost. Worth asking before optimising the operation of something that may not be required.

Where I would land

Managed, unless throughput is very large, the expertise genuinely exists in depth, and the saving is material in the context of the whole budget. And if self-managing is chosen, it should come with a named owning team, a runbook, a tested restore, and a stated review date — not as a decision made once by whoever was enthusiastic.

What a strong answer adds

Framing it as five-year TCO including the team, and noting that the calculation legitimately changes over time. The right answer at 50,000 messages a second is not the right answer at five million, and organisations rarely revisit the decision when they cross that line.