Eigenvalues, Eigenvectors, and Diagonalization

For a square matrix $A$ , an eigenvector $v \neq = 0$ satisfies $A v = λ v$ for a scalar eigenvalue $λ$ — $A$ acts on $v$ by pure scaling. Diagonalization writes $A = P D P^{- 1}$ when enough independent eigenvectors exist, turning matrix powers and exponentials into trivial scalar operations. Applies everywhere: differential equations, quantum mechanics, vibrations, PageRank, principal component analysis. Foundational result of linear algebra, crystallized in the 18th–20th centuries.

Given a linear operator on a vector space, most directions get rotated and stretched in complicated ways. Eigenvectors are the special directions that are merely scaled, and the scaling factors are the eigenvalues. Finding them reveals the intrinsic "axes" along which a transformation acts simply, and diagonalization rebuilds the whole transformation from these axes. This matters because in eigenvector coordinates a matrix becomes diagonal, so $A^{100}$ , $e^{A t}$ , and the long-term behaviour of dynamical systems become elementary. The surprising fact: a real matrix can have no real eigenvectors at all (a pure rotation), yet always has complex ones — and symmetric matrices are guaranteed to have a complete set of orthogonal real eigenvectors.

The Core Insight

A matrix is a verb — it transforms vectors. Eigenvectors are the directions on which that verb degenerates into multiplication by a number. If you change to a basis built from eigenvectors, the tangled action of $A$ unravels into $n$ independent one-dimensional scalings. Diagonalization is a change of viewpoint: the same operator looks like a diagonal matrix when you stand on its natural axes.

Formal Statement

Eigenvalue and Eigenvector

Let $A$ be an $n \times n$ matrix over a field $F$ (typically $R$ or $C$ ). A scalar $λ \in F$ is an eigenvalue of $A$ if there exists a nonzero vector $v \in F^{n}$ , the eigenvector, such that

A v = λ v, v \neq = 0.

⟳this is the defining equation; the nonzero requirement excludes the trivial

A \cdot 0 = λ \cdot 0

which holds for every

λ

Symbols: $A$ = the operator ( $n \times n$ ); $v$ = eigenvector (nonzero column); $λ$ = eigenvalue (scalar); $n$ = dimension; $F$ = scalar field.

Characteristic Polynomial and Equation

Rearranging [1] gives $(A - λ I) v = 0$ . A nonzero solution exists iff $A - λ I$ is singular, i.e.

p_{A} (λ) = det (A - λ I) = 0.

⟳a homogeneous system

M v = 0

has a nontrivial solution iff

det M = 0

; set

M = A - λ I

p_{A} (λ)

is the characteristic polynomial, a degree-

n

polynomial in

λ

. Its roots are exactly the eigenvalues. The eigenspace for

λ

E_{λ} = ker (A - λ I) = {v : A v = λ v},

⟳collect all eigenvectors for one

λ

together with

0

; this kernel is a subspace because it is a null space.

a subspace of $F^{n}$ whose nonzero members are the eigenvectors. The dimension $dim E_{λ}$ is the geometric multiplicity; the multiplicity of $λ$ as a root of $p_{A}$ is the algebraic multiplicity.

Diagonalization Theorem

An $n \times n$ matrix $A$ is diagonalizable over $F$ iff it has $n$ linearly independent eigenvectors. Equivalently, $A$ is diagonalizable iff for every eigenvalue the geometric multiplicity equals the algebraic multiplicity and $p_{A}$ splits into linear factors over $F$ . When this holds,

A = P D P^{- 1}, D = diag (λ_{1}, \dots, λ_{n}),

⟳place eigenvectors as columns of

P

; then

A P = P D

column-by-column from

A v_{i} = λ_{i} v_{i}

, and invert

P

where the columns of $P$ are the eigenvectors and the diagonal entries of $D$ are the corresponding eigenvalues in matching order.

Spectral Theorem (Real Symmetric Case)

If $A$ is real and symmetric ( $A^{⊤} = A$ ), then all eigenvalues are real and there exists an orthonormal basis of eigenvectors. Hence

A = Q D Q^{⊤}, Q^{⊤} Q = I,

⟳symmetry forces real eigenvalues and orthogonality of eigenvectors from distinct eigenvalues; Gram–Schmidt handles repeated ones.

with $Q$ orthogonal and $D$ real diagonal. Such matrices are always diagonalizable.

Why It Holds / Derivation

Diagonalization ⇔ basis of eigenvectors

Step 1. Suppose

A

has independent eigenvectors

v_{1}, \dots, v_{n}

with

A v_{i} = λ_{i} v_{i}

. Form

P = [v_{1} \dots v_{n}]

. Then

A P = [A v_{1} \dots A v_{n}] = [λ_{1} v_{1} \dots λ_{n} v_{n}] = P D .

⟳matrix–matrix product acts column-by-column; substitute each eigen-relation.

Step 2. Since the

v_{i}

are independent,

P

is invertible, so right-multiply by

P^{- 1}

A = P D P^{- 1} .

⟳from

A P = P D

, multiply both sides on the right by

P^{- 1}

Step 3 (converse). If

A = P D P^{- 1}

with

D

diagonal, then

A P = P D

; reading column

i

gives

A v_{i} = D_{ii} v_{i}

, so each column of

P

is an eigenvector, and invertibility of

P

makes them independent.

■

Eigenvectors for distinct eigenvalues are independent

Step 1. Suppose

v_{1}, \dots, v_{k}

are eigenvectors with distinct eigenvalues

λ_{1}, \dots, λ_{k}

, and assume a dependence

c_{1} v_{1} + \dots + c_{k} v_{k} = 0

with not all

c_{i}

zero; take the shortest such relation. Step 2. Apply

A

and also multiply the relation by

λ_{1}

i \sum c_{i} λ_{i} v_{i} = 0, i \sum c_{i} λ_{1} v_{i} = 0.

⟳

A

scales each

v_{i}

λ_{i}

; the second equation is the original times the scalar

λ_{1}

Step 3. Subtract:

i = 2 \sum k c_{i} (λ_{i} - λ_{1}) v_{i} = 0.

⟳the

i = 1

term cancels because

λ_{1} - λ_{1} = 0

This is a shorter relation. By minimality each $c_{i} (λ_{i} - λ_{1}) = 0$ ; distinctness gives $λ_{i} \neq = λ_{1}$ , so $c_{i} = 0$ for $i \geq 2$ , then $c_{1} v_{1} = 0$ forces $c_{1} = 0$ . Contradiction. $■$

Werner Heisenberg | 1901–1976

In 1925 Heisenberg reformulated quantum mechanics using arrays of numbers obeying non-commutative multiplication — what Born and Jordan recognized as matrices. Observables became Hermitian matrices, and the measurable values of a quantity are precisely its eigenvalues; the state collapses onto the corresponding eigenvector. The spectral theorem thus became the mathematical backbone of physical measurement, making eigenvalues among the most physically consequential objects in all of science.

Building Intuition

Picture $A$ as a rubber-sheet deformation of the plane. Most arrows you draw get both rotated and stretched. But a few special arrows only stretch — they don't swing. Those are eigenvectors; the stretch factor is the eigenvalue. A negative eigenvalue flips the arrow; $∣ λ ∣ > 1$ stretches; $∣ λ ∣ < 1$ shrinks; $λ = 0$ collapses the direction entirely (so $A$ is singular). In the eigenvector basis the deformation is just "stretch axis 1 by $λ_{1}$ , axis 2 by $λ_{2}$ " — independent one-dimensional scalings, no coupling. Diagonalization is literally the act of tilting your head until the deformation looks like simple axis-aligned stretching.

A 2×2 matrix scales its eigenvectors and bends every other vector

Worked Examples

Diagonalizing a 2×2 matrix

Given:

A = (4213)

. Step 1 — characteristic polynomial.

det (A - λ I) = (4 - λ) (3 - λ) - (1) (2) = λ^{2} - 7 λ + 10.

⟳expand the

2 \times 2

determinant

(a - λ) (d - λ) - b c

Step 2 — eigenvalues.

λ^{2} - 7 λ + 10 = (λ - 5) (λ - 2) = 0

, so

λ_{1} = 5, λ_{2} = 2

. Step 3 — eigenvectors. For

λ = 5

(A - 5 I) = (- 1 2 1 - 2)

, giving

- x + y = 0 \Rightarrow v_{1} = (1, 1)

. For

λ = 2

(A - 2 I) = (2211)

, giving

2 x + y = 0 \Rightarrow v_{2} = (1, - 2)

. Step 4 — assemble.

P = (11 1 - 2), D = (5002), A = P D P^{- 1} .

⟳columns of

P

are eigenvectors in the same order as the diagonal of

D

λ_{1} = 5, v_{1} = (1, 1); λ_{2} = 2, v_{2} = (1, - 2); A = P D P^{- 1} .

Matrix power via diagonalization

Given: the same

A

; compute

A^{4}

acting on

x_{0} = (3, 0)

. Step 1 — power rule. Since

A = P D P^{- 1}

A^{k} = P D^{k} P^{- 1}, D^{k} = diag (λ_{1}^{k}, λ_{2}^{k}) .

⟳

A^{k} = (P D P^{- 1})^{k} = P D (P^{- 1} P) D \dots P^{- 1} = P D^{k} P^{- 1}

; inner

P^{- 1} P

telescope to

I

Step 2 — coordinates. Write

x_{0}

in the eigenbasis: solve

c_{1} v_{1} + c_{2} v_{2} = (3, 0)

c_{1} + c_{2} = 3, c_{1} - 2 c_{2} = 0 \Rightarrow c_{1} = 2, c_{2} = 1

. Step 3 — scale and recombine.

A^{4} x_{0} = c_{1} λ_{1}^{4} v_{1} + c_{2} λ_{2}^{4} v_{2} = 2 \cdot 625 (1, 1) + 1 \cdot 16 (1, - 2) .

⟳each eigen-component is just multiplied by

λ^{4}

— that is the whole point of diagonalization.

A^{4} x_{0} = (1250 + 16, 1250 - 32) = (1266, 1218) .

Common Misconceptions

Distinct eigenvalues are sufficient but NOT necessary for diagonalizability. If an

n \times n

matrix has

n

distinct eigenvalues it is diagonalizable — but a matrix with repeated eigenvalues can still be diagonalizable (the identity matrix

I

has

λ = 1

repeated

n

times and is already diagonal). Repeated eigenvalues only threaten diagonalizability; they don't forbid it. Always check whether geometric multiplicity equals algebraic multiplicity. Eigenvectors are not unique, but eigenvalues are. Any nonzero scalar multiple of an eigenvector is again an eigenvector for the same

λ

(the whole line — eigenspace — qualifies). So "the eigenvector" is loose language; normalize (

∥ v ∥ = 1

) if you need a canonical choice. Also: a

0

eigenvalue is perfectly legitimate (it means

A

is singular), but the zero vector is never an eigenvector.

A non-diagonalizable matrix

The shear $A = (1011)$ has characteristic polynomial $(1 - λ)^{2}$ , so $λ = 1$ with algebraic multiplicity 2. But

(A - I) = (0010), dim ker (A - I) = 1.

⟳the only solutions of

(A - I) v = 0

are multiples of

(1, 0)

— a single eigenvector direction.

Geometric multiplicity 1 $<$ algebraic multiplicity 2, so there is no basis of eigenvectors and $A$ is not diagonalizable. Such "defective" matrices require the Jordan Canonical Form instead.

Connections

Eigen-theory threads through the entire subject. The Characteristic Polynomial packages eigenvalues as its roots, and the Cayley–Hamilton Theorem says every matrix satisfies its own characteristic equation. The Determinant equals the product of eigenvalues and the Trace equals their sum — instant sanity checks. For non-diagonalizable matrices the Jordan Canonical Form is the nearest substitute, while the Singular Value Decomposition generalizes the spectral theorem to any (even rectangular) matrix. Diagonalization is the engine behind Systems of Linear Differential Equations and the Matrix Exponential.

Real-World Applications

Quantum mechanics: observables are Hermitian operators; their eigenvalues are the only possible measurement outcomes, and eigenvectors are the corresponding pure states (energy levels from the Hamiltonian).
Principal Component Analysis (PCA): the eigenvectors of a data covariance matrix are the directions of maximal variance; eigenvalues rank their importance, enabling dimensionality reduction.
Google PageRank: web pages are ranked by the dominant eigenvector of the web's link/transition matrix — the steady-state of a random surfer.
Structural and mechanical vibration: natural frequencies of bridges, buildings, and molecules are square roots of eigenvalues of the stiffness–mass system; matching them to forcing causes resonance.
Stability of dynamical systems & control: a linear system $\overset{x}{˙} = A x$ is stable iff every eigenvalue has negative real part; eigenvalue placement is core to controller design.
Markov chains: the eigenvalue $1$ gives the stationary distribution; the second-largest $∣ λ ∣$ controls the mixing/convergence rate.
Image compression & spectral graph theory: low-rank approximations and graph clustering (the Fiedler vector) both rest on dominant or sub-dominant eigenvectors.

Historical Development

1750s — Euler & d'Alembert: encounter "principal axes" of rotating rigid bodies and oscillating systems — eigenvalues in disguise as characteristic frequencies.
1760s — Lagrange: studies the secular equation for planetary orbits, an early characteristic equation.
1829 — Cauchy: proves real symmetric ("self-adjoint") systems have real eigenvalues; foundational to the spectral theorem.
1840s — Sylvester & Cayley: develop matrix algebra; Cayley states (1858) what becomes the Cayley–Hamilton theorem.
1878 — Frobenius: rigorous theory of minimal/characteristic polynomials and canonical forms.
1904 — Hilbert: coins Eigenwert ("own/characteristic value") while studying integral operators; the German prefix eigen- survives untranslated in English.
1907 — Schur: the Schur decomposition — every complex matrix is unitarily triangularizable — underpins numerical eigenvalue methods.
1925 — Heisenberg, Born, Jordan: matrix mechanics makes eigenvalues the literal observables of physics.
1961 — Francis & Kublanovskaya: independently invent the QR algorithm, the workhorse for computing eigenvalues numerically to this day.

Subtleties & Limits

Practical eigenvalue computation is delicate. Eigenvalues can be ill-conditioned: tiny perturbations to a non-normal matrix can move them enormously (the field of pseudospectra studies this), so naively rooting the characteristic polynomial is numerically disastrous — never do it for large $n$ . The general degree- $\geq 5$ characteristic polynomial has no radical formula (Abel–Ruffini), so all robust eigenvalue methods are iterative (QR, Lanczos, Arnoldi). For very large sparse matrices, computing the full spectrum is infeasible and one settles for a few extremal eigenvalues; efficient, accurate algorithms for clustered or nearly-defective spectra remain an active research frontier in numerical linear algebra.