The Minimum Distance Property

Suppose you have a vector in three-dimensional space and a plane through the origin. You want the single point on that plane closest to your vector. Every calculus student learns to guess "drop a perpendicular." But why is that guess correct, and in what sense is it the only correct answer? This article proves, in full generality for any Inner Product Space, that the orthogonal projection of a vector onto a subspace is the unique closest point of that subspace to the vector. This fact — the Minimum Distance Property — is one of the hinges on which a great deal of applied and pure mathematics turns. It converts a geometric question ("which point is nearest?") into an algebraic one ("which point makes the difference vector orthogonal to the subspace?"). Algebraic questions can be computed. The entire theory of least-squares regression, Fourier series truncation, and signal approximation rests on exactly this theorem. The big picture is this: inner product spaces come equipped with a notion of length and angle. Hence they have a notion of distance, via d(u,v)=∥u−v∥d(u,v) = \|u-v\|. Given a subspace UU and a point vv possibly outside UU, the "distance from vv to UU" is inf⁡u∈U∥v−u∥\inf_{u \in U} \|v-u\|. The theorem asserts this infimum is achieved, achieved at exactly one point. That point is characterized by an orthogonality condition. It is not found by calculus, not by iteration, but by a clean algebraic criterion. This article builds the mental picture with a concrete plane-in-R3\mathbb{R}^3 example. It proves the theorem rigorously using the Orthogonal Decomposition already established for Orthogonal Projections. It works two nontrivial numerical examples: one in R4\mathbb{R}^4, one with polynomials. It shows exactly what breaks if orthogonality or completeness is dropped. It closes by pointing toward its most famous application: Least Squares Approximation.

Key takeaways
  • For a finite-dimensional subspace UU of an inner product space VV, every v∈Vv \in V has a unique closest point in UU.
  • That closest point is precisely the orthogonal projection PUvP_U v.
  • The minimizing property is equivalent to the orthogonality condition v−PUv⊥Uv - P_U v \perp U.
  • The proof uses the Pythagorean theorem — no calculus, no compactness arguments needed once UU is finite-dimensional.
  • Uniqueness fails without the correct inner-product geometry; existence can fail if UU is not finite-dimensional (or not closed).
  • Key idea: minimizing distance to a subspace is the same problem as decomposing a vector into a piece inside the subspace and a piece orthogonal to it.

Where this lives today

The minimum distance property is the theoretical core of Least Squares Approximation: fitting a line, plane, or polynomial to data by minimizing sum-of-squared residuals is exactly finding the orthogonal projection of a data vector onto the column space of a design matrix. In statistics, ordinary least-squares regression coefficients are computed by solving the normal equations, which are nothing but the orthogonality condition v−PUv⊥Uv - P_Uv \perp U written in coordinates. In numerical analysis, truncated Fourier series and truncated orthogonal polynomial expansions (Legendre, Chebyshev) are best L2L^2-approximations precisely because of this theorem, which is why cutting off a Fourier series at NN terms gives the provably closest trigonometric polynomial of that degree. In signal processing, projecting a noisy signal onto a subspace spanned by basis signals (denoising, compression, matched filtering) is a direct application. In computer graphics and robotics, projecting a point onto a constraint surface's tangent subspace for collision response uses the same geometry. In quantum mechanics, the "best approximation" of a state vector by a vector in a finite-dimensional subspace of a Hilbert space (e.g., truncating a basis expansion) again invokes this exact projection theorem, extended to the infinite-dimensional, complete setting via Schmidt's and Riesz's completeness arguments.

Ask the tutor: How does the minimum distance property extend to infinite-dimensional Hilbert spaces, and what role does completeness play in guaranteeing existence there?

DefinitionDistance to a subspace and best approximation

Let VV be an inner product space over F∈{R,C}\mathbb{F} \in \{\mathbb{R}, \mathbb{C}\}, and let U⊆VU \subseteq V be a subspace. For v∈Vv \in V, define the distance from vv to UU by

d(v,U)  =  inf⁡u∈U∥v−u∥.d(v, U) \;=\; \inf_{u \in U} \|v - u\|.

A vector u0∈Uu_0 \in U is called a best approximation to vv in UU (or a closest point) if

∥v−u0∥  =  d(v,U),\|v - u_0\| \;=\; d(v,U),

that is, if ∥v−u0∥≤∥v−u∥\|v - u_0\| \le \|v-u\| for all u∈Uu \in U. Recall from Orthogonal Projections that when UU is finite-dimensional, every v∈Vv \in V decomposes uniquely as

v=PUv+w,PUv∈U,w∈U⊥,v = P_U v + w, \qquad P_U v \in U,\quad w \in U^{\perp},

where PUP_U denotes the orthogonal projection of VV onto UU, and U⊥={x∈V:⟨x,u⟩=0 for all u∈U}U^\perp = \{x \in V : \langle x, u\rangle = 0 \text{ for all } u \in U\}.

Intuition

Picture R3\mathbb{R}^3 with the xyxy-plane as UU. Take a vector vv sticking up out of the plane. Any point uu in the plane and vv determine a right triangle: the leg from uu to the foot of the perpendicular PUvP_U v, the leg from PUvP_U v straight up to vv, and the hypotenuse from uu to vv. Since the vertical leg is fixed once vv is fixed, and the hypotenuse is always at least as long as either leg (Pythagoras), the hypotenuse is minimized exactly when the horizontal leg has length zero — that is, when u=PUvu = P_U v. This picture is the entire proof in miniature. The inner-product-space version replaces "plane" with any finite-dimensional subspace and "perpendicular" with the abstract orthogonal complement. Nothing about dimension 3, or about Rn\mathbb{R}^n, or about the specific inner product matters: the Pythagorean identity for orthogonal vectors is the only tool required. The figure below shows the right-triangle picture together with the algebraic decomposition v=PUv+wv = P_Uv + w.

Figure
The foot of the perpendicular is the unique closest point of UU to vv
U Pⅰv v w = v − Pⅰv u ‖v − u‖ > ‖w‖

The key algebraic reformulation to keep in mind: showing u0u_0 minimizes ∥v−u∥\|v-u\| over u∈Uu \in U is the same statement as showing v−u0∈U⊥v - u_0 \in U^\perp. Minimum-distance problems and orthogonality problems are two languages for one idea.

TheoremThe Minimum Distance Property (Best Approximation Theorem)

Let VV be an inner product space over F∈{R,C}\mathbb{F} \in \{\mathbb{R},\mathbb{C}\}, let UU be a finite-dimensional subspace of VV, and let v∈Vv \in V. Let PUvP_U v denote the orthogonal projection of vv onto UU, so that v−PUv∈U⊥v - P_U v \in U^\perp. Then: 1. (Existence and minimality) For every u∈Uu \in U,

∥v−PUv∥  ≤  ∥v−u∥.\|v - P_U v\| \;\le\; \|v-u\|.

2. (Uniqueness) If u∈Uu \in U satisfies ∥v−u∥=∥v−PUv∥\|v-u\| = \|v - P_U v\|, then u=PUvu = P_U v. In particular PUvP_U v is the unique closest point of UU to vv, so

d(v,U)=∥v−PUv∥.d(v,U) = \|v - P_U v\|.

3. (Converse characterization) Conversely, if u0∈Uu_0 \in U satisfies v−u0∈U⊥v - u_0 \in U^\perp, then u0=PUvu_0 = P_U v, and hence u0u_0 is the unique minimizer described in (1)–(2).

Proof

We use the technique of orthogonal decomposition plus the Pythagorean theorem; no compactness or calculus argument is needed because UU is finite-dimensional, which guarantees (by the theory of Orthogonal Projections) that the decomposition v=PUv+wv = P_Uv + w with w∈U⊥w \in U^\perp exists and is unique.

Step 0: the orthogonal decomposition. Since UU is finite-dimensional, it has an orthonormal basis e1,…,eke_1,\dots,e_k (Gram–Schmidt), and the orthogonal projection is defined by

PUv=∑i=1k⟨v,ei⟩ei.P_U v = \sum_{i=1}^k \langle v, e_i\rangle e_i.

Set w=v−PUvw = v - P_U v. A direct computation shows ⟨w,ej⟩=⟨v,ej⟩−∑i⟨v,ei⟩⟨ei,ej⟩=⟨v,ej⟩−⟨v,ej⟩=0\langle w, e_j\rangle = \langle v,e_j\rangle - \sum_i \langle v,e_i\rangle \langle e_i,e_j\rangle = \langle v,e_j\rangle - \langle v,e_j\rangle = 0 for each jj (using orthonormality ⟨ei,ej⟩=δij\langle e_i,e_j\rangle = \delta_{ij}), and since e1,…,eke_1,\dots,e_k span UU, linearity of the inner product in the first slot gives ⟨w,u⟩=0\langle w, u\rangle = 0 for every u∈Uu \in U. Thus w∈U⊥w \in U^\perp. This is exactly the orthogonal decomposition established in the theory of Orthogonal Projections; we take it as given input to this proof, not something to re-derive.

Step 1: existence and minimality. Let u∈Uu \in U be arbitrary. Write

v−u=(v−PUv)+(PUv−u)=w+(PUv−u).v - u = (v - P_U v) + (P_U v - u) = w + (P_U v - u).

Since PUv∈UP_U v \in U and u∈Uu \in U, the vector PUv−uP_U v - u lies in UU (subspaces are closed under subtraction). Since w∈U⊥w \in U^\perp, we have

⟨w, PUv−u⟩=0.\langle w,\, P_U v - u\rangle = 0.

Two vectors ww and PUv−uP_U v - u that are orthogonal satisfy the Pythagorean theorem:

∥v−u∥2=∥w+(PUv−u)∥2=∥w∥2+∥PUv−u∥2.\|v-u\|^2 = \|w + (P_Uv - u)\|^2 = \|w\|^2 + \|P_Uv - u\|^2.

(This is immediate from expanding ∥a+b∥2=⟨a+b,a+b⟩=∥a∥2+⟨a,b⟩+⟨b,a⟩+∥b∥2\|a+b\|^2 = \langle a+b,a+b\rangle = \|a\|^2 + \langle a,b\rangle + \langle b,a\rangle + \|b\|^2 and using ⟨a,b⟩=⟨b,a⟩=0\langle a,b\rangle = \langle b,a\rangle = 0; over C\mathbb{C} this uses conjugate-symmetry of the inner product, which still forces both cross terms to vanish when ⟨a,b⟩=0\langle a,b\rangle = 0.) Since ∥PUv−u∥2≥0\|P_U v - u\|^2 \ge 0, this identity gives

∥v−u∥2  ≥  ∥w∥2  =  ∥v−PUv∥2,\|v-u\|^2 \;\ge\; \|w\|^2 \;=\; \|v - P_U v\|^2,

and taking square roots (both sides nonnegative) yields

∥v−u∥  ≥  ∥v−PUv∥.\|v - u\| \;\ge\; \|v - P_U v\|.

Since u∈Uu \in U was arbitrary, PUv∈UP_U v \in U achieves the infimum defining d(v,U)d(v,U), proving (1) and showing the infimum is in fact a minimum: d(v,U)=∥v−PUv∥d(v,U) = \|v-P_Uv\|.

Step 2: uniqueness. Suppose u∈Uu \in U satisfies ∥v−u∥=∥v−PUv∥\|v-u\| = \|v-P_Uv\|. Reusing the identity from Step 1,

∥v−u∥2=∥w∥2+∥PUv−u∥2,\|v-u\|^2 = \|w\|^2 + \|P_Uv - u\|^2,

and substituting ∥v−u∥2=∥v−PUv∥2=∥w∥2\|v-u\|^2 = \|v-P_Uv\|^2 = \|w\|^2 gives

∥w∥2=∥w∥2+∥PUv−u∥2⟹∥PUv−u∥2=0.\|w\|^2 = \|w\|^2 + \|P_Uv-u\|^2 \quad\Longrightarrow\quad \|P_Uv - u\|^2 = 0.

Since ∥⋅∥\|\cdot\| is a norm coming from an inner product, ∥x∥=0\|x\| = 0 implies x=0x=0 (positive-definiteness of the inner product). Hence PUv−u=0P_Uv - u = 0, i.e. u=PUvu = P_Uv. This proves (2): the minimizer is unique and equals PUvP_Uv.

Step 3: converse characterization. Suppose u0∈Uu_0 \in U satisfies v−u0∈U⊥v - u_0 \in U^\perp. We must show u0=PUvu_0 = P_Uv. Consider

PUv−u0=(v−u0)−(v−PUv)=(v−u0)−w.P_Uv - u_0 = (v - u_0) - (v - P_Uv) = (v-u_0) - w.

Both v−u0v - u_0 and ww lie in U⊥U^\perp (the former by hypothesis, the latter by Step 0), and U⊥U^\perp is itself a subspace of VV (it is the kernel of the linear map x↦(⟨x,e1⟩,…,⟨x,ek⟩)x \mapsto (\langle x,e_1\rangle,\dots,\langle x,e_k\rangle), hence a subspace), so PUv−u0∈U⊥P_Uv - u_0 \in U^\perp. But also PUv−u0∈UP_Uv - u_0 \in U, since both PUvP_Uv and u0u_0 lie in UU. Therefore

PUv−u0∈U∩U⊥.P_Uv - u_0 \in U \cap U^\perp.

We claim U∩U⊥={0}U \cap U^\perp = \{0\}: if x∈U∩U⊥x \in U\cap U^\perp then ⟨x,x⟩=0\langle x,x\rangle = 0 because x∈U⊥x \in U^\perp pairs to zero against every element of UU, including xx itself (as x∈Ux \in U); positive-definiteness then forces x=0x=0. Hence PUv−u0=0P_Uv - u_0 = 0, i.e. u0=PUvu_0 = P_Uv. Combined with Steps 1–2, u0u_0 is the unique minimizer of ∥v−u∥\|v-u\| over u∈Uu \in U. ■\blacksquare

∎

Worked examples

Example 1 (a line inside R4\mathbb{R}^4, asymmetric numbers). Let V=R4V = \mathbb{R}^4 with the standard dot product, let

u1=(1,2,−1,3),u_1 = (1,2,-1,3),

and let U=span⁡(u1)U = \operatorname{span}(u_1), a one-dimensional subspace (a line through the origin). Take the target vector

v=(4,−1,2,0).v = (4, -1, 2, 0).

We want the closest point of UU to vv, and the distance from vv to UU. Step 1: normalize. Compute ∥u1∥2=12+22+(−1)2+32=1+4+1+9=15\|u_1\|^2 = 1^2+2^2+(-1)^2+3^2 = 1+4+1+9=15. So e1=115(1,2,−1,3)e_1 = \frac{1}{\sqrt{15}}(1,2,-1,3). Step 2: project. Compute ⟨v,u1⟩=4(1)+(−1)(2)+2(−1)+0(3)=4−2−2+0=0\langle v, u_1\rangle = 4(1) + (-1)(2) + 2(-1) + 0(3) = 4 - 2 - 2 + 0 = 0. Interesting: ⟨v,u1⟩=0\langle v,u_1\rangle = 0 already! Since the projection formula for a one-dimensional subspace is

PUv=⟨v,u1⟩⟨u1,u1⟩ u1=015u1=0,P_U v = \frac{\langle v,u_1\rangle}{\langle u_1,u_1\rangle}\, u_1 = \frac{0}{15}u_1 = 0,

we find PUv=0P_Uv = 0 (the zero vector, which is of course in UU). Step 3: verify orthogonality directly (checks the theorem's converse clause). Since v−PUv=v−0=vv - P_Uv = v - 0 = v, we need v∈U⊥v \in U^\perp, i.e. ⟨v,u1⟩=0\langle v, u_1\rangle = 0 — which we just verified holds. By part (3) of the theorem, this alone (without computing PUvP_U v via the formula) certifies that u0=0u_0=0 is the unique closest point. Step 4: distance. By the theorem, d(v,U)=∥v−PUv∥=∥v∥=16+1+4+0=21d(v,U) = \|v - P_Uv\| = \|v\| = \sqrt{16+1+4+0} = \sqrt{21}. Step 5: sanity check against a generic point of UU. Take u=2u1=(2,4,−2,6)∈Uu = 2u_1 = (2,4,-2,6) \in U. Then

v−u=(4−2, −1−4, 2−(−2), 0−6)=(2,−5,4,−6),v - u = (4-2,\,-1-4,\,2-(-2),\,0-6) = (2,-5,4,-6),
∥v−u∥2=4+25+16+36=81,∥v−u∥=9.\|v-u\|^2 = 4+25+16+36 = 81, \qquad \|v-u\| = 9.

Indeed 9>21≈4.5839 > \sqrt{21} \approx 4.583, consistent with u=0u=0 (not u=2u1u=2u_1) being the minimizer. We can also verify the Pythagorean bookkeeping from the proof: PUv−u=0−2u1=−2u1P_Uv - u = 0 - 2u_1 = -2u_1, so ∥PUv−u∥2=4∥u1∥2=4(15)=60\|P_Uv-u\|^2 = 4\|u_1\|^2 = 4(15)=60, and ∥w∥2=∥v∥2=21\|w\|^2 = \|v\|^2 = 21; indeed 21+60=81=∥v−u∥221 + 60 = 81 = \|v-u\|^2, matching Step 1 of the proof exactly.

Example 2 (best linear approximation to a quadratic, in C[0,1]C[0,1]). Let V=P2(R)V = \mathcal{P}_2(\mathbb{R}), polynomials of degree ≤2\le 2 on [0,1][0,1], with inner product

⟨f,g⟩=∫01f(x)g(x) dx.\langle f,g\rangle = \int_0^1 f(x)g(x)\,dx.

Let U=P1(R)=span⁡(1,x)U = \mathcal{P}_1(\mathbb{R}) = \operatorname{span}(1,x), the linear polynomials, a 2-dimensional subspace. Let

v(x)=6x2−2x+1.v(x) = 6x^2 - 2x + 1.

We seek the closest linear polynomial to vv in the L2[0,1]L^2[0,1] distance, i.e., we minimize

∫01(6x2−2x+1−(a+bx))2 dx\int_0^1 \big(6x^2-2x+1 - (a+bx)\big)^2\,dx

over real a,ba,b — and the theorem guarantees a unique minimizing pair (a,b)(a,b), obtainable purely from orthogonality, without calculus. Step 1: orthonormalize UU. Start with 11: ∥1∥2=∫011 dx=1\|1\|^2 = \int_0^1 1\,dx = 1, so e1=1e_1 = 1. Next, Gram–Schmidt on xx: subtract its e1e_1-component, ⟨x,e1⟩=∫01x dx=12\langle x,e_1\rangle = \int_0^1 x\,dx = \tfrac12. So the orthogonal remainder is x−12x - \tfrac12. Its norm: ∫01(x−12)2 dx=∫01(x2−x+14)dx=13−12+14=4−6+312=112\int_0^1 (x-\tfrac12)^2\,dx = \int_0^1 (x^2 - x + \tfrac14)dx = \tfrac13 - \tfrac12+\tfrac14 = \tfrac{4-6+3}{12}=\tfrac1{12}. So ∥x−12∥=123\|x-\tfrac12\| = \tfrac{1}{2\sqrt3}, and

e2=23(x−12).e_2 = 2\sqrt3\left(x - \tfrac12\right).

Step 2: compute ⟨v,e1⟩\langle v,e_1\rangle and ⟨v,e2⟩\langle v,e_2\rangle.

⟨v,e1⟩=∫01(6x2−2x+1) dx=6⋅13−2⋅12+1=2−1+1=2.\langle v, e_1\rangle = \int_0^1 (6x^2-2x+1)\,dx = 6\cdot\tfrac13 - 2\cdot\tfrac12 + 1 = 2 - 1 + 1 = 2.
⟨v,x−12⟩=∫01(6x2−2x+1)(x−12)dx.\langle v, x-\tfrac12\rangle = \int_0^1 (6x^2-2x+1)\left(x-\tfrac12\right)dx.

Expand (6x2−2x+1)(x−12)=6x3−3x2−2x2+x+x−12=6x3−5x2+2x−12(6x^2-2x+1)(x-\tfrac12) = 6x^3 -3x^2 -2x^2 + x + x - \tfrac12 = 6x^3 -5x^2 + 2x - \tfrac12.

∫01(6x3−5x2+2x−12)dx=6⋅14−5⋅13+2⋅12−12=32−53+1−12.\int_0^1 \big(6x^3-5x^2+2x-\tfrac12\big)dx = 6\cdot\tfrac14 - 5\cdot\tfrac13 + 2\cdot\tfrac12 - \tfrac12 = \tfrac32 - \tfrac53 + 1 - \tfrac12.

Common denominator 6: 96−106+66−36=9−10+6−36=26=13\tfrac96 - \tfrac{10}6 + \tfrac66 - \tfrac36 = \tfrac{9-10+6-3}{6} = \tfrac{2}{6} = \tfrac13. So ⟨v,x−12⟩=13\langle v, x-\tfrac12\rangle = \tfrac13, hence ⟨v,e2⟩=23⋅13=233=23\langle v,e_2\rangle = 2\sqrt3 \cdot \tfrac13 = \tfrac{2\sqrt3}{3} = \tfrac{2}{\sqrt3}. Step 3: assemble the projection.

PUv=⟨v,e1⟩e1+⟨v,e2⟩e2=2(1)+23⋅23(x−12)=2+4(x−12)=4x.P_Uv = \langle v,e_1\rangle e_1 + \langle v,e_2\rangle e_2 = 2(1) + \frac{2}{\sqrt3}\cdot 2\sqrt3\left(x-\tfrac12\right) = 2 + 4\left(x - \tfrac12\right) = 4x.

So the unique best linear approximation to v(x)=6x2−2x+1v(x) = 6x^2-2x+1 on [0,1][0,1], in the L2L^2 sense, is

PUv(x)=4x,i.e. a=0, b=4.P_Uv(x) = 4x, \qquad \text{i.e. } a=0,\ b=4.

Step 4: verify orthogonality directly (an independent check). We need v−PUv=6x2−2x+1−4x=6x2−6x+1v - P_Uv = 6x^2 - 2x + 1 - 4x = 6x^2 - 6x + 1 to be orthogonal to both 11 and xx (equivalently to 11 and x−12x - \tfrac12), since {1,x−12}\{1, x-\tfrac12\} spans UU.

∫01(6x2−6x+1) dx=2−3+1=0.✓\int_0^1 (6x^2-6x+1)\,dx = 2 - 3 + 1 = 0. \checkmark
∫01(6x2−6x+1)x dx=∫01(6x3−6x2+x) dx=32−2+12=0.✓\int_0^1 (6x^2-6x+1)x\,dx = \int_0^1 (6x^3-6x^2+x)\,dx = \tfrac32 - 2 + \tfrac12 = 0. \checkmark

Both vanish, confirming v−PUv∈U⊥v - P_Uv \in U^\perp; by part (3) of the theorem this independently proves 4x4x is the unique minimizer, without needing to trust the Gram–Schmidt arithmetic a second time. Step 5: the minimum distance. ∥v−PUv∥2=∫01(6x2−6x+1)2 dx\|v - P_Uv\|^2 = \int_0^1 (6x^2-6x+1)^2\,dx. Expand: (6x2−6x+1)2=36x4−72x3+48x2−12x+1(6x^2-6x+1)^2 = 36x^4 - 72x^3 + 48x^2 -12x+1 (check: 36x436x^4 from (6x2)2(6x^2)^2; cross terms 2⋅6x2⋅(−6x)=−72x32\cdot6x^2\cdot(-6x) = -72x^3; 2⋅6x2⋅1+(−6x)2=12x2+36x2=48x22\cdot 6x^2\cdot1 + (-6x)^2 = 12x^2+36x^2=48x^2; 2⋅(−6x)(1)=−12x2\cdot(-6x)(1) = -12x; plus 12=11^2=1). Integrate:

36⋅15−72⋅14+48⋅13−12⋅12+1=365−18+16−6+1=365−7=36−355=15.36\cdot\tfrac15 - 72\cdot\tfrac14 + 48\cdot\tfrac13 - 12\cdot\tfrac12 + 1 = \tfrac{36}{5} - 18 + 16 - 6 + 1 = \tfrac{36}{5} - 7 = \tfrac{36-35}{5} = \tfrac15.

So d(v,U)=∥v−PUv∥=15d(v,U) = \|v-P_Uv\| = \tfrac{1}{\sqrt5}, the true minimum L2L^2-distance from vv to the linear polynomials, achieved uniquely at 4x4x.

Counterexamplewhy finite-dimensionality (or at least closedness) of UU is needed

The theorem requires UU finite-dimensional (so that the orthogonal decomposition v=PUv+wv = P_Uv+w is guaranteed to exist). Drop that hypothesis, and existence of a closest point can fail entirely — not just uniqueness. Let V=ℓ2V = \ell^2, the space of square-summable real sequences (a1,a2,… )(a_1,a_2,\dots) with ⟨a,b⟩=∑nanbn\langle a,b\rangle = \sum_n a_nb_n, and let

U={ a∈ℓ2:only finitely many an are nonzero }.U = \Big\{\, a \in \ell^2 : \text{only finitely many } a_n \text{ are nonzero} \,\Big\}.

This UU is a genuine subspace of ℓ2\ell^2 (closed under addition and scalar multiplication), but it is not finite-dimensional and, more importantly, it is not closed in the ℓ2\ell^2 norm. Let v=(1,12,14,18,… )=(2−(n−1))n≥1∈ℓ2v = (1, \tfrac12, \tfrac14, \tfrac18,\dots) = (2^{-(n-1)})_{n\ge1} \in \ell^2 (this lies in ℓ2\ell^2 since ∑4−(n−1)=43<∞\sum 4^{-(n-1)} = \tfrac{4}{3}<\infty). Note v∉Uv \notin U, since every coordinate of vv is nonzero. For each NN, let uN∈Uu_N \in U be the truncation uN=(1,12,…,2−(N−1),0,0,… )u_N = (1,\tfrac12,\dots,2^{-(N-1)},0,0,\dots). Then

∥v−uN∥2=∑n>N4−(n−1)  ⟶  0as N→∞,\|v - u_N\|^2 = \sum_{n>N} 4^{-(n-1)} \;\longrightarrow\; 0 \quad \text{as } N \to \infty,

so d(v,U)=inf⁡u∈U∥v−u∥=0d(v,U) = \inf_{u\in U}\|v-u\| = 0. But no u∈Uu \in U achieves ∥v−u∥=0\|v-u\|=0, because that would force u=vu=v, and v∉Uv \notin U. So the infimum 00 is never attained — there is no closest point in UU at all, let alone a unique one. What broke, precisely? In the proof, Step 0 invoked finite-dimensionality of UU to guarantee the orthogonal decomposition v=PUv+wv = P_Uv + w exists. Here UU is infinite-dimensional and not closed, so no such PUv∈UP_Uv \in U exists; the entire argument, which is built on that decomposition, has no foothold. This shows the finite-dimensionality hypothesis (or, more generally, closedness of UU together with completeness of VV, as used in Hilbert space theory) is not a technical nicety — without it the conclusion of the theorem can fail outright, with no minimizer at all.

Scope, edge cases & common traps

SituationWhat happens
UU finite-dimensional, VV any inner product spaceTheorem applies exactly as stated: unique closest point =PUv=P_Uv.
U={0}U = \{0\}PUv=0P_Uv = 0 for all vv; distance is ∥v∥\|v\|, trivially consistent.
v∈Uv \in UThen PUv=vP_Uv = v, distance is 00, and vv is (trivially) its own unique closest point.
UU infinite-dimensional but closed and VV complete (Hilbert space)Theorem still holds, but the proof needs a compactness/completeness argument beyond finite-dimensional Gram–Schmidt (stated, proof beyond scope here).
UU a subspace that is not closed in an infinite-dimensional VVExistence can fail entirely, as shown in the counterexample above.
Working over C\mathbb{C}Everything goes through verbatim; conjugate-symmetry of ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle still forces the cross term in the Pythagorean expansion to vanish whenever the two vectors are orthogonal.

The mathematician behind it

Origins · 1 minErhard Schmidt and how this was discovered
ES
Erhard Schmidt
1876–1959 · Göttingen and Berlin, functional analysis and Hilbert space theory

Erhard Schmidt, a student of David Hilbert, is best known for co-developing the Gram–Schmidt orthogonalization process and for foundational work extending finite-dimensional Euclidean geometry into infinite-dimensional Hilbert space. His 1907–1908 papers rigorously formulated orthogonal projection and best-approximation arguments in ℓ2\ell^2-type spaces, the same geometric idea — decompose, then minimize via orthogonality — that this article proves in the finite-dimensional setting. Schmidt's insistence on treating infinite sequences of functions with the same geometric vocabulary as vectors in space was central to the birth of functional analysis.

History
- 1822: Adrien-Marie Legendre and later Gauss develop the method of least squares, implicitly minimizing distance to a subspace of "predicted values."
- 1907: Erhard Schmidt publishes work rigorously formalizing orthogonal projection and best approximation in what becomes Hilbert space.
- 1908: Schmidt's dissertation work under Hilbert crystallizes the orthogonalization process now called Gram–Schmidt (after Jørgen Gram's earlier 1883 determinant-based version).
- 1920s–1930s: von Neumann and Riesz place the projection theorem in the axiomatic framework of abstract Hilbert spaces, proving existence via completeness rather than finite bases.
- 1950s onward: the minimum distance property becomes the standard justification for least-squares estimation across statistics, numerical analysis, and engineering.
Is the closest point always found by "dropping a perpendicular," even in weird inner product spaces like polynomial spaces?

Yes — that is exactly the content of the theorem, and the perpendicularity is measured using whatever inner product the space carries, not visual right angles. In Example 2, the space was polynomials with the L2[0,1]L^2[0,1] inner product ⟨f,g⟩=∫01fg dx\langle f,g\rangle=\int_0^1 fg\,dx, and "perpendicular" meant ∫01(v−PUv)⋅u dx=0\int_0^1 (v-P_Uv)\cdot u \,dx = 0 for every u∈Uu\in U. The geometric intuition of "drop a perpendicular" survives completely; only the meaning of perpendicular changes with the inner product. This is precisely why the abstract framework of inner product spaces is powerful: one proof, phrased using ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle and ∥⋅∥\|\cdot\|, covers Rn\mathbb{R}^n, function spaces, and matrix spaces simultaneously.

Do I need to compute the full orthogonal projection PUvP_Uv to verify a candidate is the closest point?

No. Part (3) of the theorem gives a shortcut: if you can guess or construct a candidate u0∈Uu_0\in U and directly verify v−u0v-u_0 is orthogonal to every vector in a spanning set of UU, you have proven u0=PUvu_0=P_Uv is the unique minimizer, without ever building an orthonormal basis or invoking the projection formula. This is exactly what Steps 4 in both worked examples did as an independent check. In practice, this is often faster than Gram–Schmidt, especially when a natural guess for u0u_0 is available.

Why does uniqueness require the norm to come from an inner product, rather than just any norm?

The proof of uniqueness (Step 2) crucially used the identity ∥a+b∥2=∥a∥2+∥b∥2\|a+b\|^2=\|a\|^2+\|b\|^2 for orthogonal a,ba,b, which is special to inner-product norms and encodes strict convexity of the unit ball. General normed spaces (e.g. R2\mathbb{R}^2 with the max-norm ∥(x,y)∥∞=max⁡(∣x∣,∣y∣)\|(x,y)\|_\infty=\max(|x|,|y|)) can have multiple closest points to a given vector from a given subspace, because their unit balls are not strictly convex (they have flat faces). Inner product spaces avoid this because their unit balls are always strictly convex ellipsoids/spheres, which is exactly what the Pythagorean argument exploits.

How does this theorem connect to the normal equations used in linear regression?

In Least Squares Approximation, one models data as y≈Xβy \approx X\beta for a design matrix XX and seeks β\beta minimizing ∥y−Xβ∥2\|y - X\beta\|^2. Here U=Col⁡(X)U = \operatorname{Col}(X) (the column space of XX), and the minimum distance property says the minimizing XβX\beta must equal PUyP_Uy, characterized by y−Xβ⊥Col⁡(X)y - X\beta \perp \operatorname{Col}(X). Writing this orthogonality condition in coordinates as XT(y−Xβ)=0X^T(y-X\beta)=0 gives exactly the normal equations XTXβ=XTyX^TX\beta = X^Ty. So the normal equations are simply the coordinate expression of v−u0∈U⊥v - u_0 \in U^\perp from part (3) of this theorem.

What happens if vv is already inside UU?

Then vv is trivially its own closest point: taking u0=vu_0=v gives v−u0=0∈U⊥v-u_0=0\in U^\perp (the zero vector is orthogonal to everything), so by part (3) of the theorem u0=v=PUvu_0=v=P_Uv, and the minimum distance is 00. This sanity check is useful for testing whether a computed projection formula is correct: plugging in v∈Uv\in U should always return PUv=vP_Uv=v.

Does this theorem require VV itself to be finite-dimensional?

No — only UU, the subspace being projected onto, needs to be finite-dimensional. The ambient space VV can be infinite-dimensional (as in Example 2, where VV could be taken as all of C[0,1]C[0,1] rather than just polynomials, and UU is still the 2-dimensional space of linear polynomials). The proof only used finite-dimensionality of UU to build an orthonormal basis via Gram–Schmidt and hence a projection formula; VV played no special finiteness role anywhere in the argument.

What is the geometric meaning of the quantity ∥w∥2\|w\|^2 that appears throughout the proof?

w=v−PUvw = v - P_Uv is the "leftover" or residual component of vv orthogonal to UU, and ∥w∥2=∥v−PUv∥2\|w\|^2 = \|v-P_Uv\|^2 is exactly the squared minimum distance d(v,U)2d(v,U)^2. In regression language, ww is the residual vector, and ∥w∥2\|w\|^2 is the residual sum of squares — the smallest achievable total squared error when approximating vv by any element of UU.

Before this