Linear Algebra

Elimination, eigenvalues and the SVD — with the geometry kept in view.

About this course

Fourteen lectures treating linear algebra as geometry rather than bookkeeping. Elimination and the LU factorization come first, then the four fundamental subspaces, independence, basis and rank. Orthogonality leads to projections, Gram-Schmidt and least squares; determinants lead to eigenvalues, diagonalization and the spectral theorem. The final module reaches the singular value decomposition and its consequences — Markov chains, PageRank, the matrix exponential and principal component analysis.

What you will be able to do

  • Solve linear systems by Gaussian elimination and factor matrices as LU
  • Identify the four fundamental subspaces and determine independence, basis, dimension and rank
  • Compute orthogonal projections, run Gram-Schmidt, and solve least-squares problems
  • Find eigenvalues and eigenvectors, diagonalize matrices, and apply the spectral theorem to symmetric matrices
  • Compute and interpret the SVD, and apply it to PCA and Markov-chain problems

What this course covers

  • What a vector is and its two operations
  • Linear combinations and the span
  • Vector independence and the zero-combination test
  • The dot product: definition, length, and angle
  • Geometry of Rn: span in higher dimensions
  • Worked problem: dot product, length, unit vector, combination
  • Course spine: linear algebra as linear combinations
  • Determinant as Unifying Detector
  • Determinant Formulas and Cofactor Expansion
  • Geometric Meaning: Signed Area and Volume
  • Linearity in Each Column and the Collapse to Zero
  • Three Defining Properties and Key Consequences
  • Computing via Elimination: Product of the Pivots
  • Determinant as Gateway to Eigenvalues
  • Linear transformations and their defining rules
  • Geometric transformations read from matrix columns
  • Rotation matrices and length preservation
  • Change of basis and coordinate conversion
  • Similarity: same transformation, different basis
  • Kernel and image via elimination; rank–nullity
  • The transformation is basis-free; its matrix is a shadow
  • Prerequisites: spectral theorem, subspaces, orthogonal matrices
  • The SVD factorization: A = UΣVᵀ
  • Geometry of the SVD: rotate, stretch, rotate
  • Low-rank approximation and the Eckart–Young theorem
  • Four fundamental subspaces read from U and V
  • End-to-end worked SVD example
  • Clause 12 and preview of applications
  • Two pictures of a linear system
  • Gaussian elimination algorithm
  • Pivots and the three fates of a system
  • Reduced row echelon form
  • Elimination as matrix multiplication: A = LU
  • Pivot count as the master switch of invertibility
  • Full worked 3×3 elimination example
  • Orthogonality and the dot product
  • Projection onto a line
  • Projection matrix and normal equations
  • Symmetry and idempotence of projection matrices
  • Gram–Schmidt process and A = QR factorization
  • Least-squares line fit: the cold case solved
  • Lecture spine: perpendicular error as the central principle
  • Eigenvectors and eigenvalues: geometric meaning
  • The characteristic equation for finding eigenvalues
  • Complex eigenvalues and rotation
  • Spectrum, similarity invariance, and the fundamental theorem
  • Diagonalization: A = PDP⁻¹
  • Defective matrices: when diagonalization fails
  • Powers of a matrix via diagonalization
  • Dominant eigenvalue and long-run behavior
  • Eigenvalue toolkit recap
  • Markov matrices and the steady state
  • Perron-Frobenius theorem and power iteration
  • PageRank as eigenvector computation
  • The matrix exponential defined via power series
  • Eigenvalue sign and system stability
  • Fully worked matrix exponential example
  • Two guarantees of symmetric matrices
  • Spectral theorem: A = QΛQᵀ
  • Quadratic forms and their geometry
  • Positive definite matrices
  • End-to-end worked example
  • Clause eleven and module summary
  • Preview: singular value decomposition
  • The Overdetermined System and Least Squares Problem
  • Normal Equations from Perpendicular Residual
  • Why Squared Error, Not Absolute Error
  • Fitting Lines and Curves by Choosing Columns
  • R-Squared as a Score for Fit Quality
  • QR Factorization for Numerically Stable Least Squares
  • Least Squares as the Universal Data-Fitting Method
  • Course tools converging into PCA
  • Centering the data: PCA mechanics step one
  • Covariance matrix and its eigenvectors as principal components
  • Principal components as directions of maximum variance
  • Variance explained and optimal dimensionality reduction
  • Numerical stability: SVD over normal equations
  • Full worked PCA on eight-point dataset
  • Course synthesis: finding the right directions
  • Matrix–vector and matrix–matrix multiplication
  • The matrix inverse and when it exists
  • Singular matrices and the determinant as the deciding switch
  • Computing the inverse: the Gauss–Jordan sweep
  • Practical preference for LU over computing A⁻¹
  • Elimination as matrix multiplication: LU factorization
  • The matrix as operator: compose, undo, and factor
  • Vector spaces and subspaces
  • The column space and solvability
  • The null space and rank-nullity theorem
  • The four fundamental subspaces and their orthogonal pairs
  • Finding all four subspaces by hand
  • Rank as the universal size of all four subspaces
  • Linear Independence: Definition and Elimination Test
  • Spanning Sets: The Reach Test
  • Basis: Minimal Spanning and Maximal Independent Set
  • Basis as a Coordinate System and Choice of Language
  • Dimension: The Invariant Size of a Space
  • Rank: Dimension Measured for a Matrix
  • Worked Example: Finding a Basis by Elimination

Syllabus

  1. Vectors, Elimination & Subspaces

    1. Vectors, Linear Combinations & the Geometry of Rⁿ
    2. Systems of Equations & Gaussian Elimination
    3. Matrix Algebra, Inverses & the LU Factorization
    4. Vector Spaces, Subspaces & the Four Fundamental Subspaces
    5. Independence, Basis, Dimension & Rank

  2. Bases, Orthogonality & Least Squares

    1. Linear Transformations & Change of Basis
    2. Orthogonality, Projections & Gram-Schmidt
    3. Least Squares & Data Fitting

  3. Determinants, Eigenvalues & Spectral Theory

    1. Determinants
    2. Eigenvalues & Diagonalization
    3. Symmetric Matrices, Spectral Theorem & Quadratic Forms

  4. SVD, Markov Chains & PCA

    1. The Singular Value Decomposition
    2. Markov Matrices, PageRank & the Matrix Exponential
    3. Principal Component Analysis & Course Capstone