Optimization & Learning Theory
A 14-module path through optimization & learning theory, in study order from foundations to the frontier.
Modules
- Learning as Optimization
- The Loss Function
- Empirical Risk Minimization
- Convex Functions and Convex Sets
- Convex Optimization Problems
- Gradient Descent
- Learning Rates and Convergence
- Stochastic Gradient Descent
- Momentum Methods
- Regularization Theory
- PAC Learning
- VC Dimension
- The Bias-Complexity Tradeoff
- No-Free-Lunch Theorem