You are sizing the Kafka topic behind a CDC pipeline. The source Postgres takes about 12 million row changes a day across 40 tables, the average row is about 1.2 KB, the connector emits before and after images plus metadata, and the platform team wants 7 days of retention so a failed consumer can be rebuilt without a new snapshot. Roughly how much broker storage, and which assumption dominates the error?

The assumptions, stated

• 12 million changes a day.
• 1.2 KB average row, emitted twice (before and after) plus roughly 0.3 KB of envelope: source table, log position, transaction id, timestamps.
• The connector serialises to JSON, which inflates typed values into text: assume 2x over a binary encoding.
• Producer-side compression on batches: assume 3x to 5x, which is ordinary for repetitive JSON.
• Replication factor 3, which is the standard durable setting.

The arithmetic

Per change, uncompressed: 1.2 + 1.2 + 0.3 ≈ 2.7 KB, then 2x for JSON ≈ 5.4 KB.

Per day: 12,000,000 × 5.4 KB ≈ 65 GB uncompressed. At 4x compression, about 16 GB a day on disk.

Seven days: about 110 GB per replica. At replication factor 3: roughly 330 GB, with a defensible range of 250–500 GB given the compression spread.

That number is unremarkable. Three brokers with a terabyte each absorb it without a conversation. The value of the estimate is that it rules the question out, which is most of what an estimate is for.

Which assumption dominates the error

Whether before-images are emitted at all, and how wide they are. In Postgres this is set by the table's replica identity. The default emits only the primary key in the before-image; REPLICA IDENTITY FULL emits every column, which is what the 1.2 KB assumption above builds in. Getting this wrong changes the answer by about 2x in either direction, and it is a per-table setting, so an estimate that averages over 40 tables is averaging over two different regimes.

The second-largest error source is that the mean row size is the wrong statistic. If three of the 40 tables hold documents or serialised blobs, their rows may be 50 KB and they may also be the most frequently updated. A median-based estimate that ignores them will be low by a multiple. Measure the top five tables by change volume separately and treat the rest as the tail.

Update-heavy workloads matter for a third reason: log compaction cannot be used on a topic whose consumers need the full change history, so the retention is genuinely time-based and the estimate stands.

What the number rules in or out

At 330 GB, seven days of retention is free and you should take fourteen. Had it come out at 20 TB, the design changes: tiered storage that offloads older segments to object storage, or a shorter topic retention paired with a tested re-snapshot path, accepting that a long consumer outage means a rebuild rather than a replay.

One constraint sits upstream of all of this and is tighter than the topic: the source database's own write-ahead log retention. Seven days of Kafka is worth nothing if the connector falls behind the source's replication slot limit and the slot is dropped. Size the topic to your consumers' worst realistic outage; size the slot to the connector's.

When this estimate is the wrong tool

Do not do this arithmetic to choose between 7 and 14 days of retention when both are cheap. Do it to find out whether you are in the regime where retention is a design constraint at all. The cost of being wrong is asymmetric: over-provisioning storage is measured in tens of pounds a month, and under-provisioning means a consumer that cannot be rebuilt without taking a fresh snapshot of a production database, which costs a maintenance window and a conversation.

Prefer measuring to estimating whenever the pipeline already exists. One day of observed topic growth beats every assumption above, because it has already resolved the replica-identity question, the serialisation format and the long tail of wide tables. The estimate is for the pipeline you have not built yet — which is precisely when the update-to-insert ratio is unknown, and why that ratio is worth asking the source team about before writing any code. Change-data-capture connectors have behaved this way since the pattern became mainstream around 2015; none of this is new, and the planning mistake recurs anyway.