A platform team proposes one ingestion standard: every source publishes to Kafka and the warehouse loads only from Kafka. That includes 40 SaaS connectors that pull once a day. The argument is one path, one set of tools, and replayability everywhere. Review it — what would you remove, what would you keep and what would you leave alone?

What is actually required

The team wants three things: one place to look when a feed is late, one schema contract, and the ability to reprocess. All three are properties of the interface and the catalogue, not of the transport. Conflating them is the error in the proposal.

What I would remove, and why it is safe to

Put the 40 daily SaaS pulls straight into the table. A daily pull of tens of megabytes gains nothing from a log and pays for it in several currencies:

• Deployables double. Each source now needs a producer and a consumer, so 40 sources become 80 things that can fail, page someone and need upgrading.
• The failure mode gets worse, not better. The batch path fails loudly: the extract errored, and the run is red. The log path fails quietly: the consumer lags, the table is stale, and the orchestrator is green because it had nothing to run.
• Replay is already available and is better. For a source you can re-query by date, "run the extract for 2026-09-12" is auditable and idempotent. Replaying a topic reproduces what you captured, including the bug you captured it with.
• At-least-once delivery adds a deduplication problem the batch path did not have, and dedup logic in 40 consumers is 40 places to get it subtly wrong.

The one change that matters

Decide transport per source with a stated rule, and write the rule down. Put a source on the log when at least two of these hold: more than one consumer wants the stream · the producer cannot re-emit history · the freshness requirement is under roughly 15 minutes. CDC from the operational databases and the clickstream meet all three. A daily SaaS pull meets none.

What I would leave alone

The parts they were right about, which are the parts that look like bureaucracy:

• One catalogue of sources with owner, schedule, freshness commitment and downstream consumers.
• One schema contract format, enforced wherever the data lands, log or table.
• One set of pipeline observability signals, above all age-of-newest-record per dataset rather than job success.

Uniformity of interface does not require uniformity of transport, and that sentence is the whole review.

How I would argue this in the review

Not as "Kafka is overkill", which sounds like taste. As a number: 80 deployables against 40, an extra failure mode that is silent, and a replay story that is worse than the one being replaced — in exchange for an internal consistency that no consumer can observe. Then concede the real point: if the platform team's actual problem is that nobody can find out who owns a feed, the log will not fix that either, and the catalogue will.

When the proposal is not wrong

At an organisation where a second consumer appears for most datasets within a year, the uniform log stops being ceremony and starts paying. The tipping point is observable: count how many datasets have more than one consumer. Below roughly a quarter, transport uniformity costs more than it returns; above about half, the per-source decision becomes the thing that is expensive, because every new dataset restarts an argument. Choose the standard then, and choose it for the reason that actually applies rather than for replayability, which the batch path already has.