Eigenvalues & Eigenvectors

One-line: An eigenvector of a square matrix

A

is a nonzero vector whose direction is unchanged when multiplied by

A

, and its eigenvalue is the scalar by which it is stretched:

A v = λ v

. Key result in words: For an

n \times n

matrix, eigenvalues are the roots of the characteristic polynomial

det (A - λ I) = 0

; each eigenvalue has an associated subspace of eigenvectors. When enough independent eigenvectors exist, the matrix diagonalizes, turning matrix powers and exponentials into trivial scalar operations. Where it applies: Differential equations, vibrational analysis, quantum mechanics, principal component analysis, Markov chains, Google PageRank, stability theory. Who/when: Concept crystallized through Cauchy, Jacobi, and Hilbert in the 19th–early 20th century; the German eigen ("own/characteristic") was popularized by Hilbert (1904).

Given a linear operator represented by a square matrix $A$ , an eigenvector is a nonzero vector $v$ that $A$ maps to a scalar multiple of itself, and the multiplier $λ$ is the eigenvalue. These special directions reveal the intrinsic geometric action of a transformation: stripped of coordinate clutter, a complicated matrix often behaves as simple stretching along a handful of axes. Eigen-analysis matters because it converts coupled, hard problems (systems of ODEs, repeated matrix multiplication, energy functionals) into decoupled scalar problems. A surprising fact: a real matrix can have no real eigenvectors at all (a rotation rotates everything), yet the same matrix always has complex eigenvalues — algebra guarantees roots even when geometry hides them.

The Core Insight

Multiplication by a matrix usually mixes directions together. Eigenvectors are the directions the matrix refuses to mix — it only rescales them. If you choose a basis made of eigenvectors, the matrix becomes diagonal in that basis. Everything hard about a matrix (its powers, its exponential, its long-term behavior) becomes easy once you see it through its eigenstructure.

Formal Statement

Eigenvalue and Eigenvector

Let $A$ be an $n \times n$ matrix over a field $F$ (typically $R$ or $C$ ). A scalar $λ \in F$ is an eigenvalue of $A$ if there exists a nonzero vector $v \in F^{n}$ such that

A v = λ v, v \neq = 0 .

⟳This is the defining relation —

A

acts on

v

purely as scalar multiplication by

λ

. The condition

v \neq = 0

excludes the trivial solution that holds for every

λ

Such $v$ is an eigenvector associated with $λ$ . Symbols: $A$ = the matrix (linear operator); $v$ = eigenvector (direction); $λ$ = eigenvalue (scale factor); $I$ = $n \times n$ identity; $n$ = dimension.

Rearranging [1] gives the homogeneous system

(A - λ I) v = 0 .

⟳Subtract

λ v = λ I v

from both sides of [1] and factor the common

v

on the right.

A nonzero solution exists iff the coefficient matrix is singular:

det (A - λ I) = 0.

⟳A homogeneous square system

M v = 0

has a nonzero solution exactly when

M

is not invertible, i.e.

det M = 0

The left side, expanded, is the characteristic polynomial $p_{A} (λ)$ :

p_{A} (λ) = det (A - λ I) = (- 1)^{n} (λ^{n} - (tr A) λ^{n - 1} + \dots + (- 1)^{n} det A) .

⟳Expanding the determinant of

A - λ I

by cofactors yields a degree-

n

polynomial in

λ

; the leading coefficient is

(- 1)^{n}

, the

λ^{n - 1}

coefficient is

\pm tr A

, and the constant term is

det A

up to sign.

Eigenspace, Multiplicities

For an eigenvalue $λ$ , the eigenspace is the null space

E_{λ} = ker (A - λ I) = {v : A v = λ v} .

⟳The set of all eigenvectors of

λ

, together with

0

, is precisely the solution set of [2], which is a subspace.

The algebraic multiplicity $m_{a} (λ)$ is the multiplicity of $λ$ as a root of $p_{A}$ . The geometric multiplicity $m_{g} (λ) = dim E_{λ}$ . They satisfy

1 \leq m_{g} (λ) \leq m_{a} (λ) .

⟳A root contributes at least one eigenvector (

m_{g} \geq 1

), but a generalized-eigenvector argument (or the Jordan form) shows the geometric count can never exceed the algebraic count.

Diagonalizability

An $n \times n$ matrix $A$ is diagonalizable (there exists invertible $P$ with $P^{- 1} A P = D$ diagonal) iff $A$ has $n$ linearly independent eigenvectors, equivalently iff $m_{g} (λ) = m_{a} (λ)$ for every eigenvalue. In that case

A = P D P^{- 1}, D = diag (λ_{1}, \dots, λ_{n}),

where the columns of

P

are the eigenvectors and the diagonal of

D

holds the corresponding eigenvalues.

⟳Stack the relations

A v_{i} = λ_{i} v_{i}

into

A P = P D

column-by-column; invertibility of

P

(guaranteed by independence) gives

A = P D P^{- 1}

Spectral Theorem (real symmetric case)

If $A$ is real and symmetric ( $A = A^{T}$ ), then all eigenvalues are real and $A$ admits an orthonormal eigenbasis. There is an orthogonal matrix $Q$ ( $Q^{T} Q = I$ ) with

A = Q D Q^{T}, D real diagonal .

⟳Reality of eigenvalues follows from

λ ∥ v ∥^{2} = v^{*} A v = \overline{λ} ∥ v ∥^{2}

; orthogonality of eigenvectors for distinct eigenvalues comes from

(λ_{i} - λ_{j}) ⟨ v_{i}, v_{j} ⟩ = 0

Why It Holds / Derivation

From the definition to the spectrum

Step 1 — Reduce to a null-space condition. Equation [1]

A v = λ v

is equivalent to

(A - λ I) v = 0

(equation [2]). We seek a nonzero

v

. Step 2 — Singularity criterion. A square homogeneous system has a nontrivial solution iff its matrix is singular. Hence

λ

is an eigenvalue iff

det (A - λ I) = 0

(equation [3]). Step 3 — The characteristic polynomial. Expanding

det (A - λ I)

gives a monic-up-to-sign degree-

n

polynomial

p_{A} (λ)

(equation [4]). By the Fundamental Theorem of Algebra, over

C

it has exactly

n

roots counted with multiplicity, so every complex matrix has $n$ eigenvalues. Step 4 — Eigenvectors per eigenvalue. For each root

λ

, solve

(A - λ I) v = 0

. The solution set

E_{λ}

is a subspace of dimension

m_{g} (λ) \geq 1

(since

A - λ I

is singular, its null space is nontrivial). Step 5 — Independence across distinct eigenvalues. Suppose

v_{1}, \dots, v_{k}

are eigenvectors for distinct eigenvalues

λ_{1}, \dots, λ_{k}

. Assume a dependence

\sum c_{i} v_{i} = 0

with not all

c_{i} = 0

; choose a minimal such relation. Apply

A

\sum c_{i} λ_{i} v_{i} = 0

. Subtract

λ_{1} \times

(original):

\sum_{i \geq 2} c_{i} (λ_{i} - λ_{1}) v_{i} = 0

, a shorter relation — contradiction. Hence eigenvectors from distinct eigenvalues are linearly independent. Step 6 — Diagonalization. If the independent eigenvectors number

n

, they form a basis; collecting them as columns of

P

gives

A P = P D

, so

A = P D P^{- 1}

(equation [7]).

■

David Hilbert | 1862–1943

While the algebraic ideas trace to Cauchy and the term "characteristic value" was in use, it was David Hilbert who, in his 1904 work on integral equations, systematically used the prefix eigen ("own," "proper," "characteristic") — Eigenwert for eigenvalue and Eigenfunktion for eigenfunction. His spectral theory of self-adjoint operators in infinite dimensions generalized finite-dimensional eigen-analysis and laid the mathematical groundwork later seized upon by quantum mechanics, where observables are self-adjoint operators and measured values are their eigenvalues.

Building Intuition

Picture $A$ as a machine that deforms space — stretching, shearing, rotating. Most arrows get tilted and rescaled when fed through. Eigenvectors are the rare arrows that come out pointing the same way (or exactly reversed), only longer or shorter. The stretch factor is $λ$ .

$λ > 1$ : stretch along that direction.
$0 < λ < 1$ : compress.
$λ < 0$ : flip and rescale.
$λ = 0$ : collapse that direction to zero (so $A$ is singular).
complex $λ$ : rotation mixed with scaling (no real fixed direction).

If you tilt your head so your coordinate axes line up with the eigenvectors, the matrix suddenly looks diagonal — it just scales each new axis independently. That is the entire payoff of Diagonalization.

Eigenvectors of a shear-and-stretch matrix: most vectors rotate, but the eigen-directions only rescale.

Worked Examples

A 2×2 matrix with distinct real eigenvalues

Given:

A = (2112)

(symmetric). Find eigenvalues, eigenvectors, and diagonalize. Step 1 — Characteristic polynomial.

det (A - λ I) = det (2 - λ 1 1 2 - λ) = (2 - λ)^{2} - 1 = λ^{2} - 4 λ + 3.

Step 2 — Roots.

λ^{2} - 4 λ + 3 = (λ - 1) (λ - 3) = 0 \Rightarrow λ_{1} = 1, λ_{2} = 3.

Step 3 — Eigenvector for $λ_{1} = 1$ . Solve

(A - I) v = 0

(1111) v = 0 \Rightarrow v_{1} + v_{2} = 0

, so

v_{1} = (1 - 1)

. Step 4 — Eigenvector for $λ_{2} = 3$ . Solve

(- 1 1 1 - 1) v = 0 \Rightarrow v_{1} = v_{2}

, so

v_{2} = (11)

. Step 5 — Diagonalize.

P = (1 - 1 11)

D = (1003)

. Note

v_{1} \cdot v_{2} = 0

(orthogonal, as the spectral theorem guarantees).

λ_{1} = 1, v_{1} = (1, - 1)^{T}; λ_{2} = 3, v_{2} = (1, 1)^{T}; A = P D P^{- 1} .

Check:

tr A = 4 = 1 + 3

✓,

det A = 3 = 1 \cdot 3

✓ (dimensionless scalars).

A defective matrix and a matrix power

Given:

A = (3013)

. Find eigenstructure and decide diagonalizability; then compute

A^{4}

for a diagonalizable sibling. Step 1 — Eigenvalues.

det (A - λ I) = (3 - λ)^{2} = 0 \Rightarrow λ = 3

with

m_{a} = 2

. Step 2 — Eigenspace.

(A - 3 I) = (0010)

, null space spanned by

(1, 0)^{T}

only, so

m_{g} = 1

. Step 3 — Conclusion. Since

m_{g} = 1 < 2 = m_{a}

A

is not diagonalizable (it is defective; it needs a Jordan Canonical Form block). Step 4 — Powers via diagonalization (sibling). For the diagonalizable

B = (2112)

from the previous example,

B^{4} = P D^{4} P^{- 1}

with

D^{4} = diag (1^{4}, 3^{4}) = diag (1, 81)

. Using

P^{- 1} = \frac{1}{2} (11 - 1 1)

B^{4} = \frac{1}{2} (1 - 1 11) (10081) (11 - 1 1) = \frac{1}{2} (82808082) = (41404041) .

A defective (m_{g} = 1); B^{4} = (41404041) .

Common Misconceptions

" $v = 0$ is an eigenvector." Never. The zero vector satisfies $A 0 = λ 0$ for all $λ$ , which would make the concept meaningless. Eigenvectors are required nonzero. (But $λ = 0$ is a legitimate eigenvalue — it just means $A$ is singular.)
"Eigenvectors are unique." Any nonzero scalar multiple of an eigenvector is also an eigenvector for the same $λ$ ; eigenvectors are defined only up to scale (and up to a basis choice within a multi-dimensional eigenspace).
"Every matrix is diagonalizable." False — defective matrices (geometric $<$ algebraic multiplicity) are not. See the previous example.
"Real matrices have real eigenvalues." Only guaranteed for symmetric (more generally normal Hermitian) matrices. A rotation matrix has complex eigenvalues.
"Algebraic multiplicity = geometric multiplicity always." Only when the matrix is diagonalizable; equation [6] allows strict inequality.

A real matrix with no real eigenvectors

The $9 0^{\circ}$ rotation $R = (01 - 1 0)$ rotates every vector, so no real vector keeps its direction — there is no real eigenvector. Yet its characteristic polynomial is $λ^{2} + 1 = 0$ , giving complex eigenvalues $λ = \pm i$ with complex eigenvectors $(1, \mp i)^{T}$ . This shows eigenvalues live naturally over $C$ even when $A$ is real, and that real geometry can completely conceal them.

Connections

Eigen-analysis is the hub of linear algebra. The Characteristic Polynomial supplies the eigenvalues; Diagonalization uses eigenvectors to simplify a matrix; the Spectral Theorem sharpens this for symmetric/Hermitian matrices into orthogonal diagonalization. When diagonalization fails, the Jordan Canonical Form gives the next-best structure. The Singular Value Decomposition generalizes eigenvalues to all (even non-square) matrices via eigenvalues of $A^{T} A$ . In analysis, eigenvalues drive the solution of Systems of Linear Differential Equations through the Matrix Exponential, and Quadratic Forms are classified (positive-definite etc.) by the signs of eigenvalues. The Determinant equals the product and the Trace equals the sum of eigenvalues.

Real-World Applications

Google PageRank. The ranking vector is the dominant eigenvector (eigenvalue 1) of the web's stochastic link matrix; the power method iterates $x \mapsto M x$ to converge to it.
Principal Component Analysis (PCA). The directions of maximal data variance are the eigenvectors of the covariance matrix; eigenvalues give the variance captured — the backbone of dimensionality reduction in statistics and machine learning.
Structural & mechanical vibration. Natural frequencies of bridges, buildings, and aircraft are square roots of eigenvalues of $M^{- 1} K$ (mass and stiffness matrices); the Tacoma Narrows collapse is the cautionary tale of resonance with an eigen-mode.
Quantum mechanics. Observables are Hermitian operators; measured energies are eigenvalues of the Hamiltonian ( $H ψ = E ψ$ ), and stationary states are its eigenvectors.
Markov chains. The stationary distribution of a transition matrix is its left eigenvector for eigenvalue 1; the spectral gap (gap to the second eigenvalue) sets the mixing rate.
Stability of dynamical systems. A linearized system $\dot{x} = A x$ is stable iff all eigenvalues of $A$ have negative real part — the cornerstone of control theory.
Facial recognition ("eigenfaces"). Faces are projected onto leading eigenvectors of the image covariance matrix for compact, recognizable encodings.

Historical Development

1760s — Euler & Lagrange: principal axes of rotating rigid bodies and of quadratic surfaces — implicit eigen-directions.
1820s — Cauchy: proves real symmetric matrices have real "characteristic values"; introduces the term valeurs caractéristiques and the characteristic equation.
1846 — Jacobi: the iterative method for diagonalizing symmetric matrices (Jacobi rotations), still used numerically.
1858 — Cayley: matrix algebra and the Cayley–Hamilton theorem (a matrix satisfies its own characteristic polynomial).
1870 — Jordan: the Jordan canonical form, handling non-diagonalizable (defective) matrices.
1904 — Hilbert: coins/popularizes Eigenwert and Eigenfunktion in the theory of integral equations; founds spectral theory.
1925–1932 — Heisenberg, Born, von Neumann: eigenvalues become physical observables; von Neumann's Mathematical Foundations of Quantum Mechanics (1932) puts spectral theory at the center of physics.
1929 — von Mises: the power iteration method for computing dominant eigenvalues.
1961 — Francis & Kublanovskaya: the QR algorithm, the workhorse of modern numerical eigenvalue computation.

Subtleties & Limits

Conditioning and non-normality. Eigenvalues of non-normal matrices (

A A^{T} \neq = A^{T} A

) can be wildly sensitive to tiny perturbations, and eigenvectors can be nearly parallel, making numerical computation treacherous. This motivated pseudospectra (Trefethen): the set of eigenvalues reachable by small perturbations, which better predicts transient behavior than the spectrum alone. Infinite dimensions. For operators on function spaces, the spectrum need not consist of eigenvalues — there is continuous and residual spectrum (e.g., the position operator in quantum mechanics has no eigenfunctions in

L^{2}

). Generalizing eigen-analysis to unbounded self-adjoint operators is the deep content of functional analysis. Inverse eigenvalue problems. Reconstructing a matrix (or a physical structure) from prescribed eigenvalues is generally hard and not uniquely solvable — "Can one hear the shape of a drum?" (Kac, 1966) was answered no in 1992: distinct shapes can share the same Laplacian spectrum.

Eigenvalues & Eigenvectors

The Core Insight

Formal Statement

Eigenvalue and Eigenvector

Eigenspace, Multiplicities

Diagonalizability

Spectral Theorem (real symmetric case)

Why It Holds / Derivation

From the definition to the spectrum

David Hilbert | 1862–1943

Building Intuition

Eigenvectors of a shear-and-stretch matrix: most vectors rotate, but the eigen-directions only rescale.

Worked Examples

A 2×2 matrix with distinct real eigenvalues

A defective matrix and a matrix power

Common Misconceptions

A real matrix with no real eigenvectors

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also

Before this

Where this leads