An independent validator asks you to reproduce a model's training run from eight months ago. Can you? What is needed?

What the interviewer is testing

Whether you know what independent validation actually requires from the platform, which is more than most ML systems provide.

What must be versioned and retained

Training data as it was — not the current state of the source tables. This is the hardest one and the most commonly missing: if the pipeline reads live tables, the training set is unreproducible the moment the source changes. It needs an immutable snapshot or a time-travel-capable table format with retention covering the validation period.

Feature computation code, because the features are as much a part of the model as the weights, and a change to a transformation changes the model even if the training code is identical.

Training code and configuration, at the exact commit.

Hyperparameters and random seeds, or the run is not reproducible even with identical inputs.

The environment — library versions, framework version, and where relevant hardware and driver versions, since numerical differences across accelerators are real.

The resulting artifact and its evaluation results, so the reproduction can be compared.

The other two things validation requires

Implementation verification: proving the deployed artifact is the validated one. That is a signing and provenance problem — a model registry with immutable versions, and deployment by digest rather than by name.

Monitoring data: inputs and outputs logged in a form supporting drift detection, which must be decided before deployment because it cannot be reconstructed afterwards.

The honest answer for most organisations

No, not without preparation. Reproducibility has to be designed in, and the cost of retrofitting it is high — which is the argument for building it before the first high-tier model rather than after the first validation request.

What a strong answer adds

Noting the specific gap for systems built on third-party foundation models: the provider can change the model behind an endpoint without notice, which is an uncontrolled change. Pinning versions where possible, recording the exact version with every inference, and running a scheduled evaluation against a fixed set to detect provider-side change are the equivalent controls.

Common weak answers

Pointing at the model file in storage. Assuming the training notebook is sufficient.