Orthogonality and the Gram-Schmidt Process

Orthogonality is the geometric backbone of every inner-product space. Once an inner product $\langle\cdot,\cdot\rangle$ is fixed, two vectors are declared perpendicular the instant $\langle u,v\rangle=0$ — no picture of arrows required, just a number that vanishes. That single relation makes an orthonormal basis such a luxury: components become inner products, $c_i=\langle e_i,v\rangle$ ; the norm becomes a sum of squares, Parseval's identity; expansions demand no matrix inversion. The Pythagorean theorem, Fourier series, the completeness relations of quantum mechanics, and least-squares fitting are all this idea in different clothes. The Gram-Schmidt process answers the constructive question that orthonormal bases raise: given an arbitrary linearly independent set, how do we actually build one spanning the same space? The recipe is disarmingly simple: take the vectors one at a time, at each stage subtract off the part already spanned by everything built so far (an orthogonal projection), then normalize what survives; linear independence guarantees that survivor is never zero, so the machine never stalls. What comes out is nested to match the inputs: the first $k$ outputs span exactly what the first $k$ inputs did — the algebraic content of the $QR$ factorization. Run on the monomials $1,x,x^2,\dots$ , the same procedure manufactures the Legendre and Hermite polynomials that pervade mathematical physics.

Key takeaways

Two vectors are orthogonal when their inner product vanishes; an orthonormal set is mutually orthogonal with unit-length vectors, $\langle e_i,e_j\rangle=\delta_{ij}$ .
Orthonormal bases trivialize everything: components are inner products $c_i=\langle e_i,v\rangle$ , and the norm obeys Parseval, $\lVert v\rVert^2=\sum_i|c_i|^2$ .
Gram-Schmidt turns any independent set into an orthonormal one spanning the same nested subspaces — subtract each vector's projection onto all earlier outputs, then normalize.
The remainder is orthogonal to the earlier vectors by construction and nonzero by linear independence, so the process is exact and never divides by zero.
As matrices the procedure is the factorization $A=QR$ with $R$ upper triangular; the diagonal $R_{kk}=\lVert u_k\rVert$ and $\prod_k R_{kk}^2=\det G$ , the squared volume.
Run on functions with different weights it generates the classical orthogonal polynomials — Legendre, Hermite, Laguerre — the working bases of quantum mechanics.
Key idea: at each step, subtract from the next vector its orthogonal projection onto everything constructed so far, then normalize the leftover — orthogonal to all earlier vectors by construction, nonzero by linear independence.

The core insight

At each step of Gram-Schmidt, subtract from the next vector its orthogonal projection onto everything already constructed, then normalize the leftover. That leftover is orthogonal to all earlier vectors by construction, and nonzero by linear independence — so the process is both exact and guaranteed to run to completion.

Where this lives today

Gram-Schmidt is quietly everywhere a computer solves a linear problem. Its matrix form, $QR$ , is the workhorse of numerical linear algebra: LAPACK factors matrices this way (via Householder reflectors) to solve least-squares problems $\min\lVert Ax-b\rVert$ through $Rx=Q^{\dagger}b$ , sidestepping the ill-conditioned normal equations; iterated, the same factorization becomes the $QR$ algorithm for eigenvalues. In large-scale computing the modified recursion is the beating heart of Krylov-subspace methods — Arnoldi and Lanczos, behind GMRES and conjugate gradients, orthogonalize each new basis vector against its predecessors exactly as Gram-Schmidt prescribes, and "communication-avoiding" and block variants keep that stable on exascale machines. In quantum chemistry the related Löwdin orthogonalization turns overlapping atomic orbitals into an orthonormal basis; in machine learning $QR$ underlies regression and orthogonal weight initialization, and the Gram matrix reappears as the kernel of support-vector machines and Gaussian processes; even lattice cryptography's LLL reduction opens with a Gram-Schmidt step. A hand computation from 1883 now runs billions of times a second.

The physical picture

Picture building an orthogonal frame from a bundle of slanted measuring rods. Keep the first rod as your first axis — its direction is fine, only its length needs fixing. The second rod leans partly along the first, so shave off the component along axis one, and what remains sticks out perpendicular to it. For the third, shave off the two components already covered, and so on. Normalize each survivor and you have an orthonormal frame spanning the same directions as the original bundle. The "shaving off" is orthogonal projection, and the first figure freezes the decisive step: the projection of $v_2$ onto $v_1$ , and the perpendicular remainder left behind.

Figure

Gram-Schmidt in the plane: subtract from v₂ its projection onto v₁ — coefficient ⟨v₁,v₂⟩/⟨v₁,v₁⟩ = 5/10 = ½ — leaving the remainder u₂ = v₂ − ½v₁ perpendicular to v₁, which you then normalize to the orthonormal pair e₁, e₂

Why does the machine never fail? If a survivor were zero, that input would lie in the span of the earlier ones — precisely what linear independence forbids. So every remainder is nonzero and normalizable. The same picture holds for functions, where "angle" is an integral inner product $\langle f,g\rangle=\int f g\,\mathrm{d}x$ instead of a dot product. Feed Gram-Schmidt the monomials $1,x,x^2,\dots$ and it peels the lower-degree content off each power; what falls out, after the traditional rescaling, are the Legendre polynomials — the second figure shows the first three, mutually orthogonal on $[-1,1]$ .

Figure

Gram-Schmidt on the monomials 1, x, x² over the interval from −1 to 1 yields the Legendre polynomials P₀ = 1, P₁ = x, P₂ = ½(3x² − 1), mutually orthogonal because the integral of Pᵢ Pⱼ over −1 to 1 vanishes whenever i ≠ j

The core equations

Everything begins with one scalar equation. Fix an inner product; orthogonality is nothing more than its vanishing — a condition needing no coordinates, identical for arrows, matrices, and functions.

\langle u,v\rangle=0 \iff u\perp v \qquad (1)

where: $\langle u,v\rangle$ — the inner product of $u$ and $v$ ; $\perp$ — the orthogonality relation. Reading: perpendicularity is a number that vanishes, not a picture — which lets the theory transplant from $\mathbb{R}^n$ to function spaces. An orthonormal set upgrades orthogonality with unit length, packaging both into one statement via the Kronecker delta — which makes it ideal to compute in.

\langle e_i,e_j\rangle=\delta_{ij} \qquad (2)

where: $e_i$ — the $i$ -th basis vector; $\delta_{ij}$ — the Kronecker delta, $1$ when $i=j$ else $0$ . Reading: off-diagonal zeros are mutual orthogonality; the diagonal ones are unit normalization. The engine of Gram-Schmidt is the orthogonal projection — the piece of $v$ pointing along $u$ . The coefficient below makes the leftover $v-\mathrm{proj}_u v$ perpendicular to $u$ , as the first derivation proves.

\mathrm{proj}_{u}v=\frac{\langle u,v\rangle}{\langle u,u\rangle}\,u \qquad (3)

where: $\mathrm{proj}_u v$ — the component of $v$ along $u$ ; $\langle u,u\rangle=\lVert u\rVert^2$ — the squared norm of $u$ . Reading: divide the overlap by the self-overlap for a ratio, then re-attach the direction $u$ . The recursion is the algorithm itself: process the inputs in order, and for each new one strip away its projections onto the remainders already built, then normalize what is left.

u_k=v_k-\sum_{i=1}^{k-1}\frac{\langle u_i,v_k\rangle}{\langle u_i,u_i\rangle}\,u_i,\qquad e_k=\frac{u_k}{\lVert u_k\rVert} \qquad (4)

where: $v_k$ — the $k$ -th input; $u_k$ — the orthogonal remainder; $e_k$ — the $k$ -th orthonormal output. Reading: the sum removes everything already spanned; dividing by $\lVert u_k\rVert$ rescales the survivor to unit length. Once an orthonormal basis is in hand, every vector expands with no work: its components are its inner products with the basis vectors, no linear system to solve.

v=\sum_{i}\langle e_i,v\rangle\,e_i \qquad (5)

where: $\langle e_i,v\rangle$ — the $i$ -th component, the generalized Fourier coefficient of $v$ . Reading: how much of $e_i$ is in $v$ is one inner product — no other component enters. Because the basis is orthonormal, the squared length of $v$ is the sum of squared components — the Pythagorean theorem in $n$ dimensions, in physics the statement that total probability or energy is a sum over modes.

\lVert v\rVert^2=\sum_i |\langle e_i,v\rangle|^2 \qquad (6)

where: $\lVert v\rVert$ — the norm of $v$ ; $|\cdot|$ — modulus, since complex coefficients contribute $|c_i|^2$ . Reading: dropping one component costs exactly its squared coefficient — the error bound behind every truncation. Collected into matrices, Gram-Schmidt is a factorization: stack inputs as columns of $A$ and outputs as columns of $Q$ , and the coefficients and remainder-lengths assemble into an upper-triangular $R$ .

A=QR,\qquad Q^{\dagger}Q=I,\qquad R\ \text{upper triangular},\ R_{kk}>0 \qquad (7)

where: $A$ — columns are the $v_k$ ; $Q$ — columns are the orthonormal $e_k$ ; $R$ — upper-triangular matrix of overlaps and norms. Reading: "upper triangular" is the algebraic echo of "each output uses only earlier inputs" — the nested-span property. Finally, the diagonal of $R$ carries a volume: the product of the remainder-lengths is the square root of the Gram determinant, the determinant of the pairwise-inner-product matrix — the $n$ -dimensional volume of the parallelepiped the inputs span.

\det G=\det\big(\langle v_i,v_j\rangle\big)=\prod_{k=1}^{n}\lVert u_k\rVert^2 \qquad (8)

where: $G$ — the Gram matrix, $G_{ij}=\langle v_i,v_j\rangle$ ; $\lVert u_k\rVert$ — the $k$ -th remainder-length. Reading: $\det G>0$ exactly when the vectors are independent — a coordinate-free independence test and volume formula at once.

Derivation AThe projection coefficient that makes the residual orthogonal

The goal is to pin down, from first principles, the one number in equation (3): to show that the coefficient $\langle u,v\rangle/\langle u,u\rangle$ is the unique scalar leaving a residual perpendicular to $u$ , and simultaneously the scalar making $\alpha u$ the closest point to $v$ on the line through $u$ . We derive it two independent ways — minimizing a distance, and demanding orthogonality — so "the projection is the nearest point" and "the residual is orthogonal" are revealed as one fact.

Step 1. Fix $u\neq0$ and ask which multiple $\alpha u$ sits closest to $v$ : minimize the squared distance $f(\alpha)=\langle v-\alpha u,\,v-\alpha u\rangle=\lVert v\rVert^2-2\alpha\,\langle u,v\rangle+\alpha^2\lVert u\rVert^2$ (real case; the complex case replaces $\langle u,v\rangle$ by its real part, same stationary point). Why: "closest point on the line" is the honest meaning of projection, and a squared distance is a quadratic we can minimize exactly.

Step 2. Differentiate and set to zero: $f'(\alpha)=-2\langle u,v\rangle+2\alpha\langle u,u\rangle=0$ , so $\alpha=\langle u,v\rangle/\langle u,u\rangle$ ; and $f''=2\lVert u\rVert^2>0$ confirms a global minimum. Why: the positive second derivative certifies a nearest point, not a saddle — the projection genuinely minimizes the residual length.

Step 3. Derive the same $\alpha$ from orthogonality: demand the residual $r=v-\alpha u$ be perpendicular to $u$ , i.e. $\langle u,r\rangle=\langle u,v\rangle-\alpha\langle u,u\rangle=0$ , giving $\alpha=\langle u,v\rangle/\langle u,u\rangle$ again. Why: the identical coefficient from a different requirement proves "nearest point" and "orthogonal residual" are one condition — the geometric heart of the method.

Step 4. Read off $\mathrm{proj}_u v=\dfrac{\langle u,v\rangle}{\langle u,u\rangle}u$ with residual $v-\mathrm{proj}_u v$ now certified orthogonal to $u$ ; for a unit vector this collapses to $\mathrm{proj}_e v=\langle e,v\rangle\,e$ , the form used inside the recursion. Why: this is exactly the piece Gram-Schmidt subtracts at every step, so one step is proven correct before we chain them.

Go deeper · Derivation B · 1 minWhy the Gram-Schmidt output is genuinely orthonormal and never stalls

Derivation BWhy the Gram-Schmidt output is genuinely orthonormal and never stalls

The goal is to prove the two properties the method rests on: that each remainder $u_k$ is orthogonal to every earlier output, so the finished set is truly orthonormal, and that $u_k$ is never zero, so equation (4) never divides by zero. We argue by induction on $k$ , with the hypothesis that $\{e_1,\dots,e_{k-1}\}$ is orthonormal and spans the same subspace as $\{v_1,\dots,v_{k-1}\}$ , then test the new remainder against each earlier $e_j$ ; the nonzero property follows from linear independence by contradiction.

Step 1. Write the remainder using the normalized earlier vectors (legitimate since they span the same space): $u_k=v_k-\sum_{i=1}^{k-1}\langle e_i,v_k\rangle\,e_i$ , where $\langle e_i,e_i\rangle=1$ makes each coefficient simply $\langle e_i,v_k\rangle$ . Why: working in the orthonormal $e_i$ makes the next inner product collapse cleanly through the Kronecker delta.

Step 2. Take the inner product with a fixed $e_j$ , $j<k$ , using linearity: $\langle e_j,u_k\rangle=\langle e_j,v_k\rangle-\sum_{i=1}^{k-1}\langle e_i,v_k\rangle\,\langle e_j,e_i\rangle$ . Why: orthogonality to the earlier block need only be tested against each $e_j$ , and linearity pushes the product through the sum.

Step 3. Apply the induction hypothesis $\langle e_j,e_i\rangle=\delta_{ji}$ ; every term dies except $i=j$ , leaving $\langle e_j,u_k\rangle=\langle e_j,v_k\rangle-\langle e_j,v_k\rangle=0$ for all $j<k$ . Why: the subtracted projection removes exactly the $e_j$ -overlap of $v_k$ , so it cancels itself — the whole trick, made precise.

Step 4. Suppose $u_k=0$ . Then $v_k=\sum_{i<k}\langle e_i,v_k\rangle e_i$ lies in $\mathrm{span}(v_1,\dots,v_{k-1})$ , making $\{v_1,\dots,v_k\}$ dependent — a contradiction. Hence $\lVert u_k\rVert>0$ and $e_k=u_k/\lVert u_k\rVert$ is well defined, orthogonal to the earlier $e_j$ by Step 3, completing the induction. Why: this is where linear independence earns its keep — the one hypothesis that forbids a zero remainder and keeps the algorithm alive to the last vector.

Go deeper · Derivation C · 1 minGram-Schmidt is the QR factorization, and its diagonal is a volume

Derivation CGram-Schmidt is the QR factorization, and its diagonal is a volume

The goal is to show that Gram-Schmidt, in matrix form, is exactly the factorization $A=QR$ , and to extract the geometric meaning of $R$ : its diagonal entries are the successive remainder-lengths, and their product is the volume of the parallelepiped the inputs span, equal to the square root of the Gram determinant. We invert the recursion to express each input in the orthonormal basis, read the coefficients into a triangular matrix, and multiply the diagonal to recover $\det G$ — tying the algorithm to both least-squares and the geometry of volumes.

Step 1. Invert the recursion. Solving equation (4) for $v_k$ writes each input from the outputs with index $\le k$ : $v_k=\lVert u_k\rVert\,e_k+\sum_{i=1}^{k-1}\langle e_i,v_k\rangle\,e_i=\sum_{i=1}^{k}R_{ik}e_i$ , with $R_{ik}=\langle e_i,v_k\rangle$ for $i<k$ and $R_{kk}=\lVert u_k\rVert$ . Why: since $v_k$ only calls on $e_1,\dots,e_k$ , every coefficient with row above column is zero — $R$ is upper triangular, the fingerprint of nested spans.

Step 2. Stack the columns. With the $v_k$ as columns of $A$ and the $e_i$ as columns of $Q$ , the relation $v_k=\sum_i R_{ik}e_i$ is exactly the $k$ -th column of $QR$ , so $A=QR$ with $Q^{\dagger}Q=I$ , $R$ upper triangular, $R_{kk}>0$ . Why: this is the columnwise definition of matrix multiplication read backwards — algorithm and factorization are the same equations.

Step 3. Multiply the diagonal for a volume. Since $Q^{\dagger}Q=I$ , $\det G=\det(A^{\dagger}A)=\det(R^{\dagger}R)=\prod_{k=1}^{n}R_{kk}^2=\prod_{k=1}^{n}\lVert u_k\rVert^2$ . Why: the orthonormal $Q$ drops out of the determinant, so $\det G$ depends only on the remainder-lengths — the shrinking heights of the parallelepiped as each edge is measured against the flat of the earlier ones.

Step 4. Interpret. $R_{kk}=\lVert u_k\rVert$ is the distance from $v_k$ to the span of its predecessors — the $k$ -th height — and $\prod_k R_{kk}=\sqrt{\det G}$ is the volume, base times height generalized; a vanishing height means a dependent vector and a collapsed solid. Why: this makes $\det G$ a coordinate-free independence test and a volume Gram-Schmidt computes for free. *As a numerical check on the $\mathbb{R}^3$ example below: $\prod_k\lVert u_k\rVert^2=2\cdot\tfrac32\cdot\tfrac43=4=\det G$ , confirming $\det G=\prod_k\lVert u_k\rVert^2$ .*

Limiting cases

Orthogonality and Gram-Schmidt collapse to instructive special forms in several regimes, each a check on the machinery.

Case	Result	Meaning
Already-orthonormal input	$\langle v_i,v_j\rangle=\delta_{ij}\ \Rightarrow\ e_i=v_i,\ R=I$	Every projection coefficient vanishes and every norm is $1$ , so the process returns its input untouched — the identity factorization $A=Q$ .
Two vectors in the plane	$e_2=\dfrac{v_2-\langle e_1,v_2\rangle e_1}{\lVert v_2-\langle e_1,v_2\rangle e_1\rVert}$	The classic textbook step: subtract the shadow of $v_2$ on $e_1$ and normalize — the single move drawn in the first figure.
Nearly dependent input	$\lVert v_k-\sum_{i<k}\langle e_i,v_k\rangle e_i\rVert\to 0$	The remainder is a tiny difference of large vectors; catastrophic cancellation destroys orthogonality in floating point — use modified Gram-Schmidt or Householder.
Complex inner-product space	$\mathrm{proj}_e v=\langle e,v\rangle\,e,\quad \langle e,v\rangle=\overline{\langle v,e\rangle}$	The coefficient is still an inner product, but conjugation lives in a definite slot; swapping the order corrupts the result.
Function space, weight $w(x)$	$\langle f,g\rangle=\int w f g\,\mathrm{d}x$ on monomials $\to$ classical polynomials	Different weights orthogonalize $1,x,x^2,\dots$ into Legendre ( $w=1$ ), Hermite ( $w=e^{-x^2}$ ), Laguerre ( $w=e^{-x}$ ) — same algorithm, new inner product.

Common trapClassical Gram-Schmidt is numerically treacherous, and the conjugation slot is easy to flip

Two traps bite in practice. First, the classical recursion — subtracting all projections from the original $v_k$ at once — is numerically unstable: in finite precision the computed vectors quietly lose mutual orthogonality, badly so for nearly dependent inputs. The cure is modified Gram-Schmidt (subtract one projection at a time, updating in place) or Householder $QR$ ; they give the same $Q$ and $R$ in exact arithmetic but hold orthogonality far tighter in floating point. Second, in a complex space the coefficient is $\langle e_i,v\rangle$ , not $\langle v,e_i\rangle$ — the two differ by a conjugate, and mixing them corrupts the result even though the real case would never notice. Fix the convention (physics conjugates the first slot, most maths texts the second) and keep it fixed.

Worked examples

1 · A full QR factorization in ℝ³ (fully worked)

Orthonormalize $v_1=(1,1,0)$ , $v_2=(1,0,1)$ , $v_3=(0,1,1)$ with the standard dot product, and read off the factorization $A=QR$ and the volume they span.

Step 1. First vector — just normalize. $u_1=v_1=(1,1,0)$ , so $\lVert u_1\rVert^2=2$ and $\lVert u_1\rVert=\sqrt2$ . Hence $\mathbf{e_1=\tfrac{1}{\sqrt2}(1,1,0)}$ . The diagonal entry $R_{11}=\sqrt2\approx1.41421$ .

Step 2. Second vector — subtract its projection on $u_1$ . The overlap $\langle u_1,v_2\rangle=1$ over $\langle u_1,u_1\rangle=2$ gives coefficient $\tfrac12$ , so $u_2=v_2-\tfrac12 u_1=(\tfrac12,-\tfrac12,1)$ with $\lVert u_2\rVert^2=\tfrac32$ . Normalizing, $\mathbf{e_2=\tfrac{1}{\sqrt6}(1,-1,2)}$ (check $1+1+4=6$ ). Then $R_{12}=\langle e_1,v_2\rangle=\tfrac{1}{\sqrt2}\approx0.70711$ and $R_{22}=\sqrt{3/2}\approx1.22474$ .

Step 3. Third vector — subtract projections on both. The overlaps give coefficients $\langle u_1,v_3\rangle/2=\tfrac12$ and $\langle u_2,v_3\rangle/\langle u_2,u_2\rangle=(\tfrac12)/(\tfrac32)=\tfrac13$ , so

u_3=(0,1,1)-\tfrac12(1,1,0)-\tfrac13(\tfrac12,-\tfrac12,1)=(-\tfrac23,\tfrac23,\tfrac23),

with $\lVert u_3\rVert^2=\tfrac43$ . Normalizing, $\mathbf{e_3=\tfrac{1}{\sqrt3}(-1,1,1)}$ , so $R_{33}=2/\sqrt3\approx1.15470$ and $R_{23}=\langle e_2,v_3\rangle=\tfrac{1}{\sqrt6}\approx0.40825$ .

Step 4. Assemble and check. The factorization is

Q=\begin{pmatrix}\tfrac{1}{\sqrt2}&\tfrac{1}{\sqrt6}&-\tfrac{1}{\sqrt3}\\[2pt]\tfrac{1}{\sqrt2}&-\tfrac{1}{\sqrt6}&\tfrac{1}{\sqrt3}\\[2pt]0&\tfrac{2}{\sqrt6}&\tfrac{1}{\sqrt3}\end{pmatrix},\qquad R=\begin{pmatrix}\sqrt2&\tfrac{1}{\sqrt2}&\tfrac{1}{\sqrt2}\\[2pt]0&\sqrt{3/2}&\tfrac{1}{\sqrt6}\\[2pt]0&0&\tfrac{2}{\sqrt3}\end{pmatrix}.

A dot-product check gives $e_1\cdot e_2=e_1\cdot e_3=e_2\cdot e_3=0$ , and the volume $\prod_k R_{kk}=\sqrt2\cdot\sqrt{3/2}\cdot 2/\sqrt3=\mathbf{2}$ matches $\sqrt{\det G}$ and $|\det A|=2$ exactly. The three edges span a solid of volume 2, and Gram-Schmidt handed us both the orthonormal frame and that volume in one pass.

2 · Legendre polynomials from Gram-Schmidt on monomials (fully worked)

Orthonormalize $1,x,x^2$ on $L^2[-1,1]$ with $\langle f,g\rangle=\int_{-1}^{1}f g\,\mathrm{d}x$ , and recover the first Legendre polynomials.

Step 1. Zeroth function. Take $u_0=1$ , so $\langle u_0,u_0\rangle=\int_{-1}^{1}1\,\mathrm{d}x=2$ . Keeping the Legendre normalization $P_n(1)=1$ rather than unit norm, $\mathbf{P_0=1}$ .

Step 2. First function. The overlap $\langle u_0,x\rangle=\int_{-1}^{1}x\,\mathrm{d}x=0$ vanishes by odd symmetry, so nothing is subtracted: $u_1=x$ and $\mathbf{P_1=x}$ .

Step 3. Second function. Subtract projections onto both earlier remainders:

u_2=x^2-\frac{\langle u_0,x^2\rangle}{\langle u_0,u_0\rangle}\,u_0-\frac{\langle u_1,x^2\rangle}{\langle u_1,u_1\rangle}\,u_1.

The integrals are elementary: $\langle u_0,x^2\rangle=\int_{-1}^{1}x^2\,\mathrm{d}x=\tfrac23$ and $\langle u_1,x^2\rangle=\int_{-1}^{1}x^3\,\mathrm{d}x=0$ by symmetry, so only the $u_0$ term survives: $u_2=x^2-\tfrac{2/3}{2}=x^2-\tfrac13$ .

Step 4. Apply the Legendre normalization. Rescale so the value at $x=1$ is $1$ : since $u_2(1)=\tfrac23$ , multiply by $\tfrac32$ , giving $P_2=\tfrac32\big(x^2-\tfrac13\big)=\mathbf{\tfrac12(3x^2-1)}$ — the standard $P_2$ . A check confirms $\int_{-1}^{1}P_0 P_2\,\mathrm{d}x=\tfrac12(2-2)=0$ , orthogonal as promised. Gram-Schmidt on $1,x,x^2$ with the flat weight reproduces $P_0,P_1,P_2$ exactly; swapping the weight to $e^{-x^2}$ would instead deliver the Hermite polynomials.

Physics of the portrait

Gram-Schmidt is named for two people who never collaborated, working a quarter-century apart on problems that turned out to be the same — a Danish actuary counting timber, and a German student of Hilbert taming integral equations.

Origins · 1 minthe Gram-Schmidt process and how this was discovered

the Gram-Schmidt process

orthonormalization: J. P. Gram 1883, Erhard Schmidt 1907 · inner-product spaces and Hilbert-space theory

The process carries two names because it was found twice. Jørgen Pedersen Gram (1850–1916) was a Danish mathematician who spent most of his career in insurance, and it was an applied problem — approximating functions by least-squares series, once famously to model the yield of Danish forests — that led him in 1883 to the successive orthogonalization and to the determinant of pairwise inner products that now bears his name. Erhard Schmidt (1876–1959), a student of David Hilbert at Göttingen and later a founder of the Berlin school of functional analysis, wrote down the explicit recursion in 1907 inside his theory of linear integral equations, building orthonormal systems of functions in what would soon be called Hilbert space. Neither invented the idea from nothing — Laplace and Cauchy had orthogonalized functions decades earlier, and Schmidt credited Gram — but their names stuck as the algorithm entered the curriculum. The deeper lesson is that "orthogonalize, then normalize" is no trick of Euclidean geometry but a universal construction: it works in any inner-product space, finite or infinite dimensional, which is why the same three lines that straighten a bundle of arrows also manufacture the Legendre, Hermite, and Laguerre polynomials that quantum mechanics cannot do without.

History

1883

— Gram derives successive orthogonalization and the Gram determinant in a least-squares memoir

1907

— Schmidt states the explicit recursive orthonormalization in his theory of integral equations

1935

— the joint name Gram-Schmidt takes hold in the linear-algebra literature

1958

— Householder introduces reflector-based QR, a numerically stabler route to the same factorization

1967

— Björck's rounding-error analysis establishes modified Gram-Schmidt as the stable variant

Going further

Texts. Stone & Goldbart, Mathematics for Physics, §1.4–1.5 · Arfken, Weber & Harris, Mathematical Methods for Physicists, 7th ed., Ch. 5 · Hassani, Mathematical Physics, 2nd ed., Ch. 2–4 · Byron & Fuller, Mathematics of Classical and Quantum Physics, Ch. 3–4 · Riley, Hobson & Bence, Mathematical Methods for Physics and Engineering, 3rd ed., Ch. 8 · Trefethen & Bau, Numerical Linear Algebra, Lectures 7–8 (QR, Gram-Schmidt stability). Key anchor results: the projection coefficient $\alpha=\langle u,v\rangle/\langle u,u\rangle$ follows from both distance-minimization and the orthogonality condition; the $\mathbb{R}^3$ example gives $e_1=\tfrac{1}{\sqrt2}(1,1,0)$ , $e_2=\tfrac{1}{\sqrt6}(1,-1,2)$ , $e_3=\tfrac{1}{\sqrt3}(-1,1,1)$ , diagonal $R=\sqrt2,\sqrt{3/2},2/\sqrt3$ , and volume $\prod R_{kk}=|\det A|=2$ ; the Legendre integrals are $\int_{-1}^1 x^n\,\mathrm{d}x=2,0,\tfrac23,0$ and $P_2=\tfrac12(3x^2-1)$ ; and $\det G=\prod_k\lVert u_k\rVert^2$ . Historical dates for the naming and stability analysis are approximate.

Prerequisites. Inner-Product Spaces and Norms · Linear Independence, Basis, and Dimension · Vector Spaces and Subspaces. Leads to. Unitary and Orthogonal Operators · Diagonalization and the Spectral Decomposition · Adjoint Operators and Hermitian Operators · Change of Basis and Similarity Transformations.

Ask the tutor: Want me to run Gram-Schmidt on your own vectors, or show how the Hermite polynomials fall out when the weight turns Gaussian?

Does the order in which I feed the vectors to Gram-Schmidt matter?

Yes — the algorithm is order-dependent, and different orderings generally produce different orthonormal bases, though each still spans the same total space. The first input keeps its direction and only gets normalized; each later one is bent perpendicular to everything before it, so reordering yields a different $Q$ and $R$ . This matters in numerics: column-pivoted $QR$ deliberately processes the largest remainder first, improving stability and revealing the numerical rank. So the span is order-independent, but the specific basis is not.

What is the difference between classical and modified Gram-Schmidt?

They compute the same $Q$ and $R$ in exact arithmetic but behave very differently in floating point. Classical Gram-Schmidt forms all projection coefficients from the original $v_k$ and subtracts them in one batch. Modified Gram-Schmidt subtracts one projection, updates the working vector in place, then computes the next coefficient from that updated vector. Because each coefficient is taken against a partially-orthogonalized vector, rounding errors do not compound as they do in the batch, and the columns stay far closer to orthogonal — especially for nearly dependent inputs. For dense problems people often prefer Householder reflectors, which are more stable still, but modified Gram-Schmidt remains the natural choice inside iterative Krylov methods.

What happens if I accidentally feed in a linearly dependent set?

The process detects it by producing a zero remainder. At the step where $v_k$ lies in the span of its predecessors, subtracting its projections removes everything, leaving $u_k=0$ and a zero norm — normalization would divide by zero. In exact arithmetic this is the clean signal "these vectors are dependent"; you skip that vector and the survivors still form an orthonormal basis of the subspace they span. In floating point the remainder is a tiny nonzero vector, which is dangerous: dividing by its minuscule norm amplifies rounding error. Rank-revealing $QR$ handles this by flagging any remainder below a tolerance as a numerical dependency.

Why do the Legendre polynomials come out, and where do the weight functions come from?

Legendre polynomials are, by definition, the orthogonal polynomials for the flat weight $w(x)=1$ on $[-1,1]$ — and Gram-Schmidt on $1,x,x^2,\dots$ with that inner product produces exactly them, up to the rescaling $P_n(1)=1$ . Change the weight and you change the family: $w=e^{-x^2}$ gives Hermite polynomials, $w=e^{-x}$ on $[0,\infty)$ gives Laguerre, and $(1-x^2)^\alpha$ weights give Gegenbauer and Jacobi. The weights are not arbitrary — each is the natural measure of a physical problem: Hermite's Gaussian weight is the ground-state density of the harmonic oscillator, Laguerre's is tied to the hydrogen radial equation. Gram-Schmidt is the common factory; the weight is the die that stamps out which family you get.

How exactly is Gram-Schmidt the same thing as the QR factorization?

Stack your inputs as the columns of $A$ . Gram-Schmidt produces orthonormal outputs, stacked as the columns of $Q$ ; every coefficient and remainder-length you computed is an entry of an upper-triangular $R$ . The relation "each input is a combination of the outputs with equal-or-smaller index" is precisely $A=QR$ : the diagonal $R_{kk}=\lVert u_k\rVert$ records each remainder-length, the off-diagonal $R_{ik}=\langle e_i,v_k\rangle$ the overlaps you subtracted. So the algorithm and the factorization are the same equations in two notations, which is why $QR$ solvers and Gram-Schmidt are interchangeable.

Is Gram-Schmidt actually used to compute, or is it mainly a proof device?

Both, with a division of labour. As a proof device it is indispensable: it constructively shows every finite-dimensional inner-product space has an orthonormal basis, underpins the spectral theorem's orthonormal eigenbases for degenerate spectra, and builds the classical orthogonal polynomials. As a computational tool the classical form is avoided for dense problems — Householder $QR$ is more stable — but the modified form runs at enormous scale inside iterative solvers (Arnoldi, Lanczos, GMRES), orthogonalizing each freshly generated Krylov vector against the previous ones on the fly. So the textbook recursion is at once the cleanest existence proof in linear algebra and, in its modified guise, a routine that runs constantly inside simulation codes.