Copy-on-Write versus Merge-on-Read

A customer asks you to delete their record. It sits in row 4,117,332 of a 512 MB Parquet file that also holds nine million other rows. Parquet files are immutable, so there are exactly two honest options: rewrite the file without that row, or write a small note saying "row 4,117,332 is gone" and make every future reader consult the note.

Those two options are copy-on-write and merge-on-read, and every lakehouse table format offers both. The choice is not about correctness. Both produce the same query results. It is about which side of the workload absorbs the cost.

Copy-on-write pays at write time

Under copy-on-write, a DELETE or UPDATE reads every data file that contains an affected row, rewrites it with the change applied, and commits a new snapshot pointing at the replacements. Readers are untouched: they scan plain Parquet with no merge step, and query planning stays simple.

The amplification is brutal for small changes. Deleting one row from a 512 MB file writes 512 MB. A nightly job updating 50,000 rows scattered across 400 files rewrites every one of those 400 files, which might be 200 GB of I/O to change a few megabytes of logical data. Write amplification here is the ratio of bytes written to bytes logically changed, and copy-on-write drives it to four or five orders of magnitude on point updates.

Copy-on-write is the right default when writes are large and batched, when updates are clustered (a partition rewrite rather than scattered rows), and when read latency is the thing being optimised.

Merge-on-read pays at read time

Merge-on-read leaves the data file alone and writes a delete file beside it. Two flavours existed under the Iceberg v2 specification. Position deletes name a data file and the ordinal positions of dead rows inside it, which is precise and cheap to apply. Equality deletes name predicate values, such as user_id = 88213, which lets a streaming writer emit a delete without first knowing where the row lives, at the cost of forcing readers into an anti-join against the delete file.

Now the reader does work on every scan. It must find the delete files that apply to each data file, load them, and filter. Under high churn this becomes the dominant cost, and a table with thousands of accumulated delete files can be slower to read than the same data with no deletes at all.

Deletion vectors collapse the tradeoff

Iceberg format version 3 replaces v2 position delete files with binary deletion vectors: a compact roaring bitmap, stored in a Puffin file, with at most one deletion vector per data file, marking which row positions are dead. The reader tests a bitmap bit per row rather than joining against a delete file, and because the format guarantees a single vector per data file, the count cannot grow without bound the way delete files did. Format version 3 reached general availability in Apache Iceberg 1.10.0, with engine support rolling out through 2025 and 2026; Delta Lake shipped an equivalent deletion-vector mechanism earlier.

This does not abolish the tradeoff, it moves it. Deletion vectors make merge-on-read reads much cheaper, which makes merge-on-read viable for far more tables, but the rows are still physically present, still read off disk, and still decompressed before being discarded. A table where 40% of rows are deleted still pays to read 100% of the bytes.

When it breaks

Merge-on-read without compaction degrades without warning. The delete state is unbounded until something rewrites it away. A table that performs well for two weeks and then falls off a cliff is almost always accumulating deletes faster than compaction removes them.

Equality deletes are the expensive kind. They are convenient for CDC ingestion because the writer does not need to locate rows, but they force a join and they apply to every data file in scope rather than one. Converting equality deletes to position deletes, or to deletion vectors, is a standard maintenance job for a reason.

Copy-on-write and concurrency interact badly. A long-running rewrite of 400 files holds an optimistic transaction open across all of them. Any concurrent writer touching the same files loses the commit race and must retry the whole rewrite, and under steady concurrent load a copy-on-write job can fail to make progress at all.

The engine, not the table, often decides. Read and write modes are per-table properties (write.delete.mode, write.update.mode, write.merge.mode in Iceberg), but engine support for each combination is uneven, and a writer that does not understand deletion vectors will either refuse the table or fall back silently. Check what every engine touching the table actually implements before choosing.