Model-Free Prediction & Control
A 17-module path through model-free prediction & control, in study order from foundations to the frontier.
Modules
- Monte Carlo Prediction
- First-Visit vs Every-Visit Monte Carlo
- Monte Carlo Control
- Exploring Starts and Epsilon-Soft Policies
- Temporal-Difference Learning: TD(0)
- Bootstrapping and the Bias-Variance Tradeoff
- On-Policy vs Off-Policy Learning
- SARSA: On-Policy TD Control
- Q-Learning: Off-Policy TD Control
- Expected SARSA
- Maximization Bias and Double Learning
- Importance Sampling for Off-Policy Methods
- n-Step Bootstrapping Methods
- TD(lambda) and Eligibility Traces
- Linear Function Approximation
- Semi-Gradient Methods with Function Approximation
- The Deadly Triad