Under what circumstances would you self-manage a message broker rather than use a managed service, and how would you justify it?
Show the full answer Hide the answer
What is being tested
Whether your default is managed and whether you can articulate specific rather than general reasons to deviate.
The default position
Managed, unless there is a named reason. The premium is usually less than a fraction of an engineer's time, and operational capacity is the scarcest resource in most organisations.
A message broker is a particularly strong case for managed, because the operational surface is large — brokers, partitions, quotas, rebalancing, upgrades, disk management, and the specific knowledge of why a consumer group is stuck at 3am.
The legitimate reasons to self-manage
1. A required capability the managed version does not expose. A specific version, a plugin, a configuration the provider locks down, an unusual topology.
2. Cost at material scale. At very high throughput the managed premium can become significant. This is a calculation, and the threshold is much higher than teams assume — and the calculation must include the run effort on the other side.
3. Regulatory or residency constraints the provider cannot satisfy.
4. Existing deep expertise and a team whose actual job is running this infrastructure. Not one enthusiastic engineer — a team, with a rotation.
5. Unacceptable failure modes. A documented failover behaviour or maintenance window that conflicts with your requirements.
6. Avoiding a hard dependency where the broker's availability is more critical than the provider's SLA offers.
How to justify it financially
Compare total cost of ownership over three years, converted to cost per unit of business value:
| Term | Managed | Self-managed |
|---|---|---|
| Service or licence | High | Low or zero |
| Infrastructure | Included | Real, including redundancy to match |
| Build effort | Days | Weeks to months |
| Run effort | Near zero | Ongoing — the dominant term |
| On-call burden | Provider's | Yours |
| Opportunity cost | None | What those engineers would have built |
| Risk-adjusted incident cost | Provider's maturity | Your first year will not match it |
The run effort is what is always omitted. Half an engineer's ongoing attention is a substantial annual cost that appears on no invoice.
What managed does not remove
Your responsibility for your own resilience. Failover is documented behaviour, not an invisible one; the application must tolerate a broker connection reset. Backups may exist and are almost never restore-tested by you.
"Managed" means somebody else does the operations, not that the operations do not happen — and reading the failure behaviour before adopting is the step most often skipped.
What a strong answer adds
Noting the most cited counter-example — a very large storage workload moved off public cloud — and what made it correct: one component dominating the entire cost base, at enormous scale, with a uniform workload and a team built to operate it. Those conditions are narrow, and citing that case as general support for self-hosting proves much less than it appears to.