Eigenvalues and Eigenvectors

Given a linear transformation represented by a square matrix $A$, an eigenvector is a nonzero vector whose direction is unchanged by $A$, and its associated eigenvalue is the factor by which it is stretched, compressed, or flipped. These special directions reveal the intrinsic geometry of the transformation, decoupling a complicated map into independent one-dimensional scalings. They matter because countless physical and computational problems — coupled oscillators, Markov chains, quantum observables, data covariance — become trivially solvable once expressed in an eigenbasis. A surprising fact: a real matrix can have complex eigenvalues (encoding rotation), and a perfectly innocent-looking matrix may fail to have enough eigenvectors to form a basis at all (defectiveness).

The Core Insight

Most vectors get knocked off their original line when you apply a matrix. Eigenvectors are the rare vectors that stay on their own line — the matrix only rescales them. If you can find enough of these invariant directions to build a basis, the matrix becomes diagonal in that basis, and applying it (or its powers, or its exponential) reduces to scalar arithmetic on each axis. Eigen-analysis is therefore the search for the coordinate system in which a linear map is as simple as possible.

Formal Statement

Eigenvalue and Eigenvector

Let $A\in {\mathbb{C}}^{n\times n}$ be a square matrix (or a linear operator $T:V\to V$ on a vector space $V$). A scalar $\lambda \in \mathbb{C}$ is an eigenvalue of $A$ if there exists a nonzero vector $\mathbf{v}\in {\mathbb{C}}^n$ such that

Here $A$ is the $n\times n$ matrix, $\lambda$ is the eigenvalue (a scalar), $\mathbf{v}$ is the corresponding eigenvector, $I$ is the $n\times n$ identity matrix, and $0$ is the zero vector. The vector $\mathbf{v}$ is said to belong to $\lambda$.

Rewriting $[1]$ as $(A-\lambda I)\mathbf{v}=0$ shows that $\mathbf{v}$ lies in the null space of $A-\lambda I$. For a nonzero solution to exist, $A-\lambda I$ must be singular:

Characteristic Polynomial, Spectrum, Eigenspace

The function $p_A(\lambda )=\det (A-\lambda I)$ is the characteristic polynomial of $A$; it is a degree-$n$ polynomial in $\lambda$. Its roots are precisely the eigenvalues, and the multiset of roots is the spectrum $\sigma (A)$. For each eigenvalue $\lambda$, the set

is a subspace called the eigenspace of $\lambda$. Its dimension is the geometric multiplicity $g(\lambda )=\dim E_{\lambda }$, while the multiplicity of $\lambda$ as a root of $p_A$ is the algebraic multiplicity $a(\lambda )$.

Multiplicity Inequality and Diagonalizability

For every eigenvalue $\lambda$ of $A$,

A matrix $A$ is diagonalizable (i.e. $A=PD{P}^{-1}$ with $D$ diagonal) if and only if $g(\lambda )=a(\lambda )$ for every eigenvalue, equivalently iff $A$ possesses $n$ linearly independent eigenvectors.

Trace and Determinant from Eigenvalues

Counting eigenvalues ${\lambda }_1,\dots ,{\lambda }_n$ with algebraic multiplicity,

Why It Holds / Derivation

Eigenvalues are roots of the characteristic polynomial

Eigenvectors for distinct eigenvalues are linearly independent

David Hilbert | 1862–1943

Though eigenvalue ideas predate him, it was David Hilbert who, around 1904 in his work on integral equations, coined the German prefix eigen ("own," "characteristic," "proper"). He generalized the finite-dimensional spectral picture to infinite-dimensional function spaces, introducing what we now call Hilbert spaces and laying the spectral foundations that John von Neumann would later use to axiomatize quantum mechanics. Hilbert's insight — that an operator's "own values" govern the resonant frequencies of physical and analytic systems — turned eigenanalysis from a computational trick into a structural principle of modern mathematics and physics.

Building Intuition

Picture a linear map as a deformation of a rubber sheet: vectors get stretched, sheared, and rotated. Almost every arrow you draw will point in a new direction after the map. Eigenvectors are the special arrows that survive the deformation pointing the same way — they lie along the "grain" of the transformation. The eigenvalue tells you what happens along that grain: $\lambda >1$ stretches, $0<\lambda <1$ compresses, $\lambda <0$ flips, and $|\lambda |=1$ preserves length.

If you find enough independent grains to span the whole space, you can describe the map by saying simply "stretch by ${\lambda }_1$ along axis 1, by ${\lambda }_2$ along axis 2, …" — that is diagonalization. Complex eigenvalues $\lambda =r{e}^{i\theta }$ signal that the map rotates by angle $\theta$ while scaling by $r$; there is then no real invariant line, which is why a 2D rotation has no real eigenvectors.

A shear matrix scaling its eigenvector while a generic vector turns

Worked Examples

A symmetric 2×2 matrix

Note the eigenvectors are orthogonal (dot product $=0$), as guaranteed for symmetric matrices.

Complex eigenvalues from a rotation-scaling matrix

Common Misconceptions

A defective matrix

Consider $A=\left(\begin{array}{cc}2& 1\\ 0& 2\end{array}\right)$. Its characteristic polynomial is $(2-\lambda {)}^2$, so $\lambda =2$ with algebraic multiplicity $a=2$. But solving $(A-2I)\mathbf{v}=\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)\mathbf{v}=0$ forces $v_2=0$, giving the single eigenvector direction $(1,0{)}^{\top }$. Hence $g(2)=1<2=a(2)$: there are not enough eigenvectors to span ${\mathbb{R}}^2$, so $A$ is not diagonalizable. The best one can do is the Jordan Canonical Form $\left(\begin{array}{cc}2& 1\\ 0& 2\end{array}\right)$ — a Jordan block.

Connections

Eigen-analysis is the hub of linear algebra. It underlies Diagonalization (writing $A=PD{P}^{-1}$), generalizes to the Jordan Canonical Form when diagonalization fails, and refines into the Singular Value Decomposition for non-square or non-diagonalizable maps (via eigenvalues of $A^{\top }A$). For symmetric/Hermitian matrices, the Spectral Theorem guarantees a real spectrum and an orthonormal eigenbasis. The Matrix Exponential $e^{At}=P{e}^{Dt}{P}^{-1}$ solves linear Systems of Differential Equations, while Markov Chains and PageRank hinge on the dominant eigenvalue. The characteristic polynomial ties eigenvalues to the Cayley–Hamilton Theorem, and to the Determinant and Trace through equation $[5]$.

Real-World Applications

1. Quantum mechanics. Observables are Hermitian operators; their eigenvalues are the only measurable values (energy levels of an atom = eigenvalues of the Hamiltonian), eigenvectors the stationary states.
2. Google PageRank. A web page's importance is the dominant eigenvector of the web's stochastic link matrix; the algorithm computes it by power iteration on billions of pages.
3. Principal Component Analysis (PCA). The eigenvectors of a data covariance matrix are the principal axes; their eigenvalues quantify variance, enabling dimensionality reduction in machine learning and statistics.
4. Structural & mechanical vibration. Natural frequencies of bridges, buildings, and molecules are square roots of eigenvalues of stiffness/mass matrices; resonance disasters (e.g. Tacoma Narrows analysis) trace to these modes.
5. Stability of dynamical systems. A fixed point of $\dot{\mathbf{x}}=A\mathbf{x}$ is stable iff every eigenvalue of $A$ has negative real part — the cornerstone of control theory.
6. Facial recognition (Eigenfaces). Faces are projected onto eigenvectors of an image covariance matrix, compressing recognition to a few coordinates.
7. Quantum chemistry & spectroscopy. Molecular orbital energies and vibrational spectra are computed as eigenvalues of large Hamiltonian/Hessian matrices.

Historical Development

• 1743 — Euler. While solving systems of linear ODEs, Leonhard Euler encounters "characteristic" exponents, implicitly using eigenvalue ideas.
• 1760s — Lagrange. In studying the secular (long-term) motion of planetary orbits, Joseph-Louis Lagrange analyzes the "secular equation" — the characteristic equation in disguise.
• 1820s — Cauchy. Augustin-Louis Cauchy systematizes the theory of quadratic forms and proves that symmetric matrices have real "characteristic roots," an early Spectral Theorem.
• 1846 — Jacobi. Carl Jacobi develops iterative methods to diagonalize symmetric matrices, foreshadowing numerical eigenvalue algorithms.
• 1858 — Cayley. Arthur Cayley founds matrix algebra and (with Hamilton) the Cayley–Hamilton theorem linking a matrix to its characteristic polynomial.
• 1870s — Jordan & Weierstrass. Camille Jordan introduces the canonical form for defective matrices; Karl Weierstrass develops elementary divisors theory.
• 1904 — Hilbert. David Hilbert coins Eigenwert/Eigenfunktion and extends spectral theory to infinite dimensions in his work on integral equations.
• 1925–1932 — Heisenberg, von Neumann. Eigenvalues become the measured quantities of quantum mechanics; von Neumann gives the rigorous spectral theorem for unbounded operators.
• 1960s — Francis & Kublanovskaya. The QR algorithm is invented, making large-scale numerical eigenvalue computation practical and underpinning modern scientific computing.

Subtleties & Limits

See Also