Introduction to Machine Learning

What it means for a machine to learn, and how the classical methods work

About this course

A first course in machine learning: supervised and unsupervised methods, how models are evaluated and optimised, and how to choose between them.

What you will be able to do

  • Frame a task as a learning problem and choose an appropriate model class
  • Fit and interpret linear, logistic and kernel-based models
  • Evaluate models honestly, including generalisation and model selection
  • Apply clustering and dimensionality reduction to unlabelled data
  • Explain how a neural network is trained by backpropagation

What this course covers

  • K-means algorithm and its convergence guarantee
  • Clustering objective as definition, not proxy
  • Choosing K: elbow method and silhouette score
  • Initialization sensitivity and K-means++ seeding
  • Model mismatch: when cluster shape assumptions fail
  • EM algorithm for Gaussian mixture models
  • Hierarchical clustering and the curse of dimensionality preview
  • Field Notes: Four Building Blocks
  • The Perceptron Rule and Convergence Theorem
  • Linear Separability, XOR, and the Wall
  • Non-linearity as the Prerequisite for Depth
  • Backpropagation as the Chain Rule Applied Right to Left
  • Vanishing Gradients and the Choice of Activation Function
  • Clause 13 and the Lecture's Central Takeaway
  • K-Nearest Neighbors algorithm mechanics
  • Feature scaling as part of the model definition
  • Why training accuracy cannot select K
  • Choosing K with held-out data and cross-validation
  • Curse of dimensionality: distance concentration
  • High-dimensional volume geometry
  • KD-tree speed and its high-dimensional failure
  • The Machine Learning Paradigm Shift
  • Supervised Learning Vocabulary and Goal
  • Nearest Neighbor Classifier
  • Train vs. Test Evaluation Protocol
  • Bias–Variance Tradeoff and Model Capacity
  • Course Spine Clause One and Practitioner's Loop
  • The Accuracy Trap on Imbalanced Data
  • The Confusion Matrix
  • Precision, Recall, F1, and the Threshold Trade-off
  • ROC Curve Construction
  • AUC as a Ranking Probability
  • Calibration and the Brier Score
  • Leakage and the Fifth Clause of the Spine
  • Regression and the squared-error loss
  • The linear model: parameters W and B
  • Normal equations: closed-form solution
  • Gradient descent
  • Feature scaling and condition number
  • Convexity and the course spine
  • Label-noise ceiling on accuracy
  • Decision tree structure and properties
  • Entropy and information gain for splitting
  • Depth as the capacity dial and overfitting
  • Bagging and variance reduction via disagreement
  • Boosting with weak learners in sequence
  • Eighth spine clause and ensemble diagnostics
  • Weight Initialization and Variance Propagation
  • Batch Normalization as Scale Enforcement
  • Epistemic Status of Empirical Deep-Learning Practices
  • Dropout and the Inverted-Dropout Correction
  • Convolutional Layers and the Weight-Sharing Assumption
  • Transfer Learning and Regime Dependence
  • Aggregate Accuracy and Hidden Subgroup Disparities
  • PCA as SVD of centered data
  • What a principal component is
  • Choosing the number of components
  • Reconstruction error as cost of compression
  • Variance ≠ usefulness (the cruel dataset)
  • t-SNE as a neighbourhood-preserving drawing
  • Clause 12 and the projection habit
  • Condition number and the canyon loss surface
  • Gradient descent stability ceiling and step-count ledger
  • Momentum: gradient memory and the heavy-ball method
  • Adaptive per-coordinate step sizes (Adam)
  • Stochastic gradients and the noise floor
  • Learning-rate schedules and convexity
  • Worked arithmetic: why the two rates tie exactly
  • Maximum-Margin Boundary and Support Vectors
  • Soft Margin and the Role of C
  • Hinge Loss and Its Zero-Charge Property
  • The Kernel Trick
  • Arithmetic Verification of Key Numbers
  • Ninth Clause of the Spine
  • Five Takeaways and Course Outlook
  • Unpruned tree and 1-NN equivalence
  • Bias-variance-noise decomposition
  • Capacity and the bias-variance trade-off
  • Ridge regularization measured in components
  • Seven classifiers on three datasets
  • Fit time vs. predict time (two clocks)
  • Choosing a method by interrogating the data
  • Fifteenth clause and course synthesis
  • Polynomial features and model capacity
  • Training error as a measure of flexibility, not quality
  • Overfitting and the train/validation split
  • The validation-set selection problem and the three-way split
  • Regularization: Ridge and Lasso
  • Cross-validation for model selection
  • The third clause of the spine and the exchange-rate summary
  • Failure of linear regression for classification
  • The sigmoid function and its properties
  • Score as log-odds
  • Log-loss and why squared error fails for classification
  • Gradient of log-loss through sigmoid equals P − Y
  • Decision boundary, weight interpretation, and separable-data pathology
  • Softmax for multi-class classification
  • Naïve Bayes model and independence assumption
  • Zero-frequency problem and Laplace smoothing
  • Maximum likelihood vs. maximum a posteriori estimation
  • Generative vs. discriminative contest and parameter estimation cost
  • Bayes error and the irreducible floor
  • Odds-form arithmetic for Naïve Bayes
  • Tenth clause of the course spine

Syllabus

  1. What Learning Is, and Linear Models

    1. L1 · What Is Learning?
    2. L2 · Linear Regression
    3. L3 · Beyond Straight Lines
    4. L4 · Logistic Regression and Classification

  2. Evaluation and Optimisation

    1. L5 · Evaluating Models
    2. L6 · Optimization for Machine Learning

  3. Classical Methods

    1. L7 · k-Nearest Neighbours and the Curse of Dimensionality
    2. L8 · Decision Trees and Ensembles
    3. L9 · Support Vector Machines and Kernels
    4. L10 · Probabilistic Models

  4. Unsupervised Learning

    1. L11 · Unsupervised Learning — Clustering
    2. L12 · Unsupervised Learning — Dimensionality Reduction

  5. Neural Networks and Practice

    1. L13 · Neural Networks I — From the Perceptron to Backpropagation
    2. L14 · Neural Networks II and Machine Learning in Practice
    3. L15 · The Learning Toolkit — When To Use What