A recommendation platform stores petabytes of interaction events used for both ad-hoc analysis and model training, with occasional need to delete individual users' data. Which storage architecture fits, and why does the deletion requirement drive the choice?

Why the deletion requirement is decisive

A plain data lake stores immutable files. Deleting one user's rows means finding every file containing them, rewriting each without those rows, and atomically swapping them — across petabytes, with concurrent readers and writers. Without a table format providing transactions and file-level metadata, this is either impossible or a bespoke, error-prone engineering project repeated for every erasure request.

An open table format (Iceberg, Delta, Hudi) supplies exactly the missing pieces:

• Row-level deletes, via delete vectors or merge-on-read, without rewriting entire partitions immediately.
• Snapshot isolation, so readers see a consistent view while files are being rewritten.
• Metadata that maps values to files, so a deletion touches the files that contain the user rather than scanning everything.
• Compaction that applies accumulated deletes efficiently on a schedule rather than synchronously.

Why the other options fail

Plain lake. Excellent for cost and throughput, and it cannot do targeted deletion, schema evolution or concurrent writes safely. Compliance requirements alone rule it out for user-level data.

Traditional warehouse for raw events. Storage cost at petabyte scale is prohibitive, and it does not serve the machine-learning access pattern — training reads whole columns of billions of rows directly from files, which a warehouse's query interface makes slow and expensive.

Transactional database. Wrong by orders of magnitude on both cost and analytical throughput.

What the lakehouse gives beyond deletion

• One copy of the data serving analysis and training, which avoids the divergence that comes from maintaining two pipelines — the origin of training/serving skew.
• Schema evolution without rewriting history, essential when event schemas change constantly and historical data must remain readable.
• Time travel, which makes training reproducible: a model can be retrained against the exact snapshot it originally saw.
• Engine independence. Multiple processing engines read the same tables, so the storage decision does not lock in the compute decision.

The costs to state honestly

Compaction and metadata maintenance are real, continuous operational work — neglected, small-file accumulation destroys query performance. Metadata itself becomes large enough to need its own tuning. Deletes are not free: merge-on-read shifts cost to query time, and heavy delete volume degrades reads until compaction catches up. And the ecosystem is younger than either lakes or warehouses, so operational maturity varies.

The transferable point

The interesting requirement in a data architecture is rarely the volume — it is usually the one constraint that eliminates the simple answer. Here, petabytes alone would justify a plain lake. Targeted deletion is what forces transactional metadata, and recognising which requirement is doing the eliminating is the actual architectural skill.