Lakehouse Table Format
A metadata layer over object storage that adds atomic commits, snapshots and schema evolution to files that otherwise have none.
Object storage gives durability and cheap capacity, and nothing else. A directory of Parquet files has no transactions, so a reader can see a half-written update; no schema history, so a column rename is a rewrite; and no way to ask what the table looked like yesterday.
Iceberg, Delta and Hudi all solve this the same way: a metadata layer that tracks which files constitute the table at each version. A commit swaps one metadata pointer atomically, so readers see either the old set or the new one and never a mixture. Snapshots make time travel and rollback ordinary operations.
Two consequences matter architecturally. Engine independence becomes real — several query engines can read the same table without copying it — which is the main reason to prefer an open format over a proprietary one. And maintenance becomes mandatory: compaction, snapshot expiry and orphan file cleanup are scheduled jobs, and a table with no such jobs degrades in both cost and performance until someone notices.