Feature Engineering Strategies
Practice the feature engineering moves that win interviews and ship in production: missing data handling, categorical encoding, nonlinear transforms, and interaction features—*without leaking labels*.
Students can: - build leakage-safe feature pipelines with sklearn, - choose appropriate encoding strategies, - add meaningful interaction features, - validate that features help via disciplined evaluation.
Progress — 0/7 tasks
Interview Angles
- • When do interaction features help linear models?
FAANG Gotchas
- • Imputation can hide a strong signal: missingness itself.
Asked At
Feature Engineering Strategies — Student Lab (Titanic)
Focus: missing data, encoding, nonlinear transforms, and interactions — leakage-safe.
Section 0 — Load Titanic (Kaggle) with fallback
Expected: data/titanic/train.csv
If missing, a tiny synthetic dataset is used so the notebook runs.
Baseline pipeline
Section 1 — Baseline pipeline
Task 1.1: Train a baseline with leakage-safe preprocessing
Use:
- ●
Numeric: Age, Fare, SibSp, Parch
- ●
Categorical: Pclass, Sex, Embarked
- ●
Create X/y
- ●
Build ColumnTransformer (impute + scale numeric, impute + one-hot categorical)
- ●
Evaluate LogisticRegression with StratifiedKFold CV
Checkpoint: Why is the pipeline necessary to avoid leakage in CV?
Missing data: indicators
Section 2 — Missing data: indicators
Task 2.1: Add missingness indicator features
Add: Age_is_missing, Cabin_is_missing
- ●Create these columns
- ●Add them to numeric pipeline
- ●Re-run CV and compare
Checkpoint: Why can missingness itself be predictive?
Encoding: One-hot vs Target encoding
Section 3 — Encoding: One-hot vs Target encoding
Task 3.1: Extract Title from Name and one-hot encode
Example titles: Mr, Mrs, Miss, Master.
- ●Extract Title with regex
- ●Add it as categorical feature
- ●Re-run CV
Task 3.2: Naive target encoding (demonstrate leakage)
We will intentionally do something wrong: compute mean Survived per Title using the full dataset, then map it back.
- ●Create Title_target_mean using full data (this is leakage)
- ●Compare CV score and explain why it inflates
Checkpoint: How do you fix target encoding correctly?
Nonlinear transforms + binning
Section 4 — Nonlinear transforms + binning
Task 4.1: Log-transform Fare
Interaction features
Section 5 — Interaction features
Task 5.1: FamilySize and IsAlone
FamilySize = SibSp + Parch + 1 IsAlone = FamilySize == 1
Task 5.2: Sex × Pclass cross
Create a categorical cross feature like Sex_Pclass.