Layer Contract

Bronze, silver and gold are frequently adopted as a naming convention: tables are placed in a layer by whoever created them, with no agreed meaning. A "silver" table may be raw and a "gold" table may be a staging step.

The distinguishing property is the guarantee, not the position. Silver guarantees a schema and a quality level; gold guarantees a definition; bronze guarantees only fidelity to the source.

If a consumer cannot rely on that, the layers are decoration.

Why it matters

The layers exist so that a consumer can choose the right trade between fidelity and convenience without reading every pipeline. Without stated guarantees, every consumer must inspect the table to find out what it actually contains — which is the situation the layering was supposed to remove.

Implementation patterns

• State the guarantee per layer and enforce it with tests: silver tables have a declared schema, no duplicates on the business key, and defined null semantics.
• Bronze is immutable and reproducible. Its value is that any downstream error is corrected by reprocessing rather than re-fetching from a source that may no longer have the data.
• Ownership per table, since a dataset with no owner is one nobody fixes.
• Lineage, so a wrong number in a dashboard is traceable to its inputs.
• Compaction as a platform responsibility, because small-file proliferation from streaming writers slows every reader and no individual writer is responsible for it.
• Treat consumers reading from bronze as a product signal. It means silver does not serve them, and a consumer on bronze is coupled to the source's raw schema and will break when the source changes — so the response is to extend silver rather than to accept the bypass.

Industry example

Lakehouse platforms such as Databricks popularised the layering, and the organisations that get value from it are the ones that treat each layer as a contract with tests rather than as a folder structure. The tell is simple: ask an analyst what they can assume about a silver table. If the answer is "it depends which one", the layers are names.

Failure scenarios

• Layers as folders, conveying nothing.
• Consumers on bronze, coupled to source schemas.
• Gold tables with no owner, diverging in definition from each other.
• No lineage, so a wrong number cannot be traced.
• Compaction unowned, so read performance degrades for everyone with no responsible party.

Trade-offs

Enforcing guarantees means pipelines fail when data does not meet them, which is disruptive and is the point — a pipeline that silently loads bad data into a layer that claims quality has broken the contract everything downstream depends on.

The alternative is quarantining bad records and continuing, which keeps the pipeline running and requires someone to act on the quarantine. Either is defensible; not deciding is not, since the default is usually silent coercion.

And three layers is a convention rather than a law — some platforms need two and some four, and the number matters far less than whether each has a guarantee.

Interview question

"Your analyst is about to build a report on a silver table. What can they assume, and how would they verify it without reading the pipeline?"