Feature Engineering
Encoding, missingness, selection, target leakage, and the train-serve skew that eats offline gains.
7concepts
145flashcards
51minutes of reading
- 01 Categorical Encoding Why one-hot encoding breaks down at high cardinality, how target encoding trades that for a leakage risk it must then defend against, and what each choice assumes about unseen categories.
- 02 Feature Selection: Filter, Wrapper and Embedded Methods Filter, wrapper and embedded methods trade cost against how much of the model they consult, and all three produce wildly optimistic accuracy if the selection step happens outside the cross-validation loop.
- 03 Missing Data Mechanisms Whether imputation is safe depends on why the value is absent, and the three-way distinction between MCAR, MAR and MNAR decides which methods are valid and which quietly bias the result.
- 04 Target Leakage The failure mode where a feature encodes information unavailable at prediction time, why cross-validation cannot detect it, and the three structural forms it takes.
- 05 Train-Serve Skew The gap between how a feature is computed in a training pipeline and how it is computed in a serving path, which silently degrades a model that was never wrong in offline evaluation.