Feature Engineering
Encoding, missingness, selection, target leakage, and the train-serve skew that eats offline gains.
7concepts
145flashcards
51minutes of reading
- 01 Feature Scaling and Transformations Standardisation, power transforms and quantile maps change what distance, penalties and gradients see, so they matter enormously for some model families and not at all for others, and transforming the target quietly changes what the model predicts.
- 02 Categorical Encoding Why one-hot encoding breaks down at high cardinality, how target encoding trades that for a leakage risk it must then defend against, and what each choice assumes about unseen categories.
- 03 Feature Selection: Filter, Wrapper and Embedded Methods Filter, wrapper and embedded methods trade cost against how much of the model they consult, and all three produce wildly optimistic accuracy if the selection step happens outside the cross-validation loop.
- 04 Missing Data Mechanisms Whether imputation is safe depends on why the value is absent, and the three-way distinction between MCAR, MAR and MNAR decides which methods are valid and which quietly bias the result.
- 05 Target Leakage The failure mode where a feature encodes information unavailable at prediction time, why cross-validation cannot detect it, and the three structural forms it takes.
- 06 Train-Serve Skew The gap between how a feature is computed in a training pipeline and how it is computed in a serving path, which silently degrades a model that was never wrong in offline evaluation.
- 07 Class Imbalance, Resampling and Reweighting Rebalancing classes by oversampling, SMOTE or class weights mostly moves the decision threshold while destroying probability calibration, and the evidence suggests strong learners gain little from it compared with choosing the threshold from actual costs.