An interviewer says - a regulator can ask us what any published report said on any date in the past two years and why it said that. Our platform already has a Data Vault. Where do you take this?
Show the full answer Hide the answer
What the interviewer is testing
Whether you treat reproducibility as a modelling property. It is not. A number is reproducible only when three things are pinned together: the data version, the code version and the parameters - and a Data Vault pins exactly one of them.
The clarifying questions that change the answer
- "What did the report say" or "what was true as at that date"? The first is an archive question, the second is a bitemporal query. Most regulators ask the first and most engineers answer the second.
- What is the restatement policy? If figures may be corrected after publication, every number needs two dates - the business date and the version date - and the report must say which it is showing.
- How long must this hold? Two years rules out relying on time travel, which is typically 7 to 90 days.
- Who signs the number? The signature defines what must be reproducible, and it is usually a rendered PDF of 12 figures, not a 400-model lineage graph.
A strong answer's arc
The vault gives you load-dated satellites, so the state of a business key at any past load time is recoverable. That is the data version, and it is genuinely valuable. Then name what it does not cover:
- Code. The transformation that turned hubs and satellites into the published figure has changed many times in two years. Reproducing July's number needs July's SQL, which means the run records the git commit it used and the orchestrator can check out and execute an old commit against old data. Few platforms can do this, and the ones that can tested it.
- Parameters and reference data. FX rates, hierarchies, product mappings, segment definitions. This is where reproducibility breaks in practice, because reference tables are usually small, overwritten in place and owned by someone outside engineering. They need the same versioning as the facts.
- The archive. Store the published output - the rendered figures with their as-at date and the commit that produced them. Crude, cheap, and the only mechanism that answers "what did it say" with certainty.
Then sequence it: the archive first because it is a week of work and it discharges most of the obligation, the reference-data versioning second because it is the real gap, and the reproduce-on-demand path third, tested quarterly on a random past date so you discover its breakage before a regulator does.
Common weak answers
- "We have a Data Vault, so we are auditable." The vault records what the source said. The regulator is asking what you published, which is downstream of code the vault knows nothing about.
- "Time travel covers it." 7 to 90 days against a two-year obligation, and it reproduces data, not logic.
- "We keep all the logs." Logs show that a job ran. They do not let you re-derive 1.4 million rows.
- Proposing full bitemporal modelling across the warehouse. Correct and enormous. Scope it to the tables the signed reports use - usually under ten.
What a strong answer adds
The cost line: an archive of rendered outputs for two years is megabytes; versioned reference data is a modelling change in a handful of tables; the reproduce-on-demand path is the expensive one and it is only needed if the regulator can ask "why", not just "what". Say which of the three you would not build, and why that is defensible - that is the judgement being assessed.