Experiment Tracking & Reproducibility
Run metadata, seeds and determinism, environment capture, and what it takes to rebuild a result a year later.
5concepts
60flashcards
35minutes of reading
- 01 Data Versioning and Content Addressing Why versioning datasets is harder than versioning code, how content addressing makes it tractable without copying, and what a dataset version has to mean to be useful.
- 02 Hyperparameter Search Hygiene Why random search beats grid search, how early-stopping schedulers change the budget calculation, and the selection bias that makes the best run's reported score an overestimate.
- 03 Seeds and the Limits of Determinism Why setting a seed does not make a training run reproducible, the specific sources of non-determinism on a GPU, and how to decide how much determinism is worth paying for.