Neural Network Foundations
A 16-module path through neural network foundations, in study order from foundations to the frontier.
Modules
- Biological Inspiration for Neural Networks
- The Artificial Neuron
- The Perceptron
- Activation Functions: Sigmoid, Tanh, ReLU
- The Multilayer Perceptron
- The Universal Approximation Theorem
- Forward Propagation
- Loss Functions for Deep Networks
- Computational Graphs
- The Chain Rule for Neural Networks
- Backpropagation
- Automatic Differentiation
- The Vanishing and Exploding Gradient Problem
- Weight Initialization Strategies
- Xavier and He Initialization
- The Neural Network Training Loop