This is the capstone. Six lessons reframed trading as something supervised learning can never be: a sequential decision problem, played against a market that reacts to every move you make. You learned the formal language of a Markov decision process — state, action, reward, transition, policy, return, and the discount factor that sets your horizon — and the one feature that separates a market from a video game: your action moves the environment. You learned that reward design is the hardest, most dangerous part of applied RL, because the agent optimizes the literal scalar you write, not the goal you meant — so a reward without risk and cost terms breeds a reckless overtrader, and any loophole becomes a reward hack. You learned the two great algorithm families — value-based (Q-learning, DQN, with experience replay and target networks fighting the deadly triad) and policy-gradient (REINFORCE, actor–critic, PPO) — and why continuous actions and exploration that costs real spread push trading toward policy methods and offline RL. You saw the canonical win, optimal execution, where RL re-derives Almgren–Chriss under its assumptions and earns its keep beyond them, and the perilous craft of market making, where inventory skew, the Avellaneda–Stoikov reservation price, and adverse selection collide with a reward function full of perverse incentives. And you ended on the lesson that humbles all the rest: the sim-to-real gap, where an agent that prints money in a flawed simulator — overfitting its own chosen data — loses it live. No formula sheet, no hints, no take-backs: every answer locks the instant you submit, the wrong options are the exact traps that fool real desks, and your score stays hidden until the end.
Big picture
Reinforcement Learning for Trading — the whole ladder
- Reinforcement Learning for Trading
- Markets as an MDP
- State, action, reward, transition, policy, return
- Discount factor γ sets the horizon
- Control not prediction: the action moves the environment
- Reward design & pitfalls
- One scalar per step: risk-adjusted, cost-aware
- γ too low = myopic overtrading
- Potential-based shaping; beware reward hacking
- Value vs policy methods
- Q-learning, Bellman, TD; DQN replay + target nets
- Policy gradient / REINFORCE: continuous, high variance
- Actor–critic & PPO; explore costs spread → offline RL
- Optimal execution
- Impact vs timing risk; Almgren–Chriss front-loads
- State = inventory + time + signal; reward = −cost − λ·risk
- RL re-derives AC, adds signal/nonlinear impact/constraints
- Market making
- Capture the spread; skew quotes against inventory
- Avellaneda–Stoikov reservation price; adverse selection
- Reward ignoring inventory = blow-up; perverse incentives
- The sim-to-real gap
- Agent learns the simulator, exploits its flaws
- RL overfits worse: chooses its own data + huge search
- Realistic fills, multi-regime, walk-forward, kill-switch
- Markets as an MDP
One run, one shot
This exam is graded and irreversible. Each question locks the moment you submit it — there is no Back button, no retry, and no Restart. A wrong answer simply fails that question and the exam moves on. Your pass/fail score appears only at the very end. Read every option before you commit.
Why is trading a fundamentally poor fit for plain supervised learning, even with a great return forecast?
Select an answer to continue.
Where this sits on the ladder
Passing this exam closes out the reinforcement-learning rung of the quant ladder — the point where control theory, deep learning, and a hard-won skepticism fuse into an agent that can act in a market rather than merely forecast it. You now own the discipline that separates a working RL trader from the graveyard of beautifully-converged backtests: frame the problem as an MDP, design a reward that can’t be hacked, pick the method family that fits your action space, respect the impact-versus-risk trade-off in execution, balance inventory against adverse selection when making markets, and — above all — assume your simulator is lying until reality says otherwise. The deepest lesson echoes every course before it: the algorithm is never your biggest risk; the human who trusted a flawless simulation always is.