This is the capstone. Six lessons taught the brutal craft of pointing machine learning at financial markets — where the signal-to-noise ratio is so vile that off-the-shelf ML, applied naively, produces nothing but confident, well-cross-validated, completely fictitious alpha. You learned why finance breaks the usual ML assumptions: the signal-to-noise is tiny (IC ≈ 0.02–0.05, R² near zero), the data-generating process is non-stationary (regimes shift, edges decay, concepts drift), and the analyst’s own repeated experiments make the whole exercise a multiple-testing minefield. You learned to build stationary-yet-memory-preserving features with fractional differentiation and to label outcomes the right way with the triple-barrier method and meta-labeling instead of naive fixed-horizon returns. You learned the seven faces of data leakage and why ordinary k-fold cross-validation is a lie on time series — and how purging and embargoing (and combinatorial purged CV) repair it. You learned why the best of N backtests is a near-guaranteed mirage, how the expected maximum Sharpe under the null grows like √(2 ln N), and how the deflated Sharpe ratio and probability of backtest overfitting put a real hurdle under a result. You learned the bias–variance logic behind regularization, why bagging kills variance while boosting chases bias (and overfits noise), and why MDI lies in-sample while MDA tells the out-of-sample truth — and that importance is never causation. Finally, you learned to fold a weak ML prediction into a real book: prediction → position, combining orthogonal alphas via IR ≈ IC·√breadth, respecting capacity, turnover and decay, and deploying with a locked test set, paper trading, live-IC monitoring and a kill-switch. No formula sheet, no hints, no take-backs: every answer locks the instant you submit, the wrong options are the exact traps that fool real desks, and your score stays hidden until the end.
Big picture
Machine Learning for Alpha — the whole ladder
- Machine Learning for Alpha
- ML & the overfitting enemy
- Signal-to-noise is tiny: IC ≈ 0.02–0.05, R² near zero
- Non-stationarity: regimes shift, edges decay, concept drift
- Human-in-the-loop multiple testing → in-sample ≠ out-of-sample
- Features & labeling
- Fractional differentiation: stationary BUT keeps memory
- Triple-barrier: upper/lower/vertical, first touch, vol-scaled
- Meta-labeling + uniqueness weights for overlapping labels
- Leakage & purged CV
- Leakage: overlap, look-ahead, preprocess-on-all, survivorship
- k-fold lies on time series even without shuffling
- Purge overlapping labels + embargo a band; CPCV gives many paths
- Backtest overfitting & deflated Sharpe
- Best of N inflates: E[max Sharpe] ≈ √(2 ln N)·σ_trials
- PSR adjusts for track length, skew, kurtosis
- DSR & PBO; min backtest length grows with ln N
- Models, ensembles & importance
- Bias–variance; regularize (ridge/lasso, depth, early stop)
- Bagging ↓variance (random forest) vs boosting ↓bias (overfits noise)
- MDI biased in-sample vs MDA permutation; importance ≠ causation
- From ML signal to portfolio
- Prediction → position: proportional, rank L/S, confidence, Kelly-lite
- Combine orthogonal alphas: IR ≈ IC·√breadth; capacity & turnover
- Decay & retrain; lock test set, paper trade, monitor IC, kill-switch
- ML & the overfitting enemy
One run, one shot
This exam is graded and irreversible. Each question locks the moment you submit it — there is no Back button, no retry, and no Restart. A wrong answer simply fails that question and the exam moves on. Your pass/fail score appears only at the very end. Read every option before you commit.
A vendor pitches an ML equity model with an out-of-sample R² of 0.01 and a cross-sectional IC around 0.04. What is the correct reaction?
Select an answer to continue.
Where this sits on the ladder
Passing this exam closes out the machine-learning rung of the quant ladder — the point where statistics, software, and a hard-won skepticism fuse into a process that can extract a real (if fragile) edge from data designed to fool you. You now own the discipline that separates working ML alpha from the graveyard of beautifully cross-validated backtests: respect the low signal-to-noise, label and validate without leakage, deflate every result for the trials behind it, prefer importance that survives out-of-sample, and fold the signal into a sized, monitored, kill-switchable book. The deepest lesson echoes the rest of systematic trading — the model is never your biggest risk; the human running it, fooled by a great-looking backtest, always is.