Orthogonality and Bessel's Inequality

The projection/optimality argument underlying Bessel's inequality is the prototype for Hilbert space theory in functional analysis: the same three-line completed-square proof, verbatim, establishes that orthogonal projection onto any closed subspace of an inner product space minimizes distance, with Bessel's inequality as the general statement that $\sum |\langle f,e_n\rangle|^2 \le \|f\|^2$ for any orthonormal system $\{e_n\}$, trigonometric or not (e.g. Legendre polynomials, Hermite functions, wavelets). In signal processing and engineering, this is exactly the statement that a signal's "energy" bounds the total power captured by any finite set of orthogonal frequency channels, underlying spectral analysis, the design of orthogonal transforms (DCT/DFT used in JPEG and MP3 compression), and the theory of orthogonal filter banks. In quantum mechanics, Bessel's inequality (and its equality form, Parseval/Plancherel) governs expansions of a wavefunction in an orthonormal basis of eigenstates, guaranteeing that probabilities computed from projection coefficients never exceed total probability $1$ after normalization. In statistics, the same least-squares argument is literally the derivation of ordinary least-squares regression: fitting a linear model by minimizing squared error is orthogonal projection onto the column space of the design matrix, and the "residual sum of squares decomposition" is Pythagoras in disguise. In numerical analysis and approximation theory, Bessel's inequality certifies a priori that truncated orthogonal expansions (Chebyshev, Legendre, spherical harmonics) cannot have unbounded coefficient growth, which underlies convergence and stability estimates for spectral methods used in solving partial differential equations. Finally, the Riemann–Lebesgue lemma derived here is a basic tool in the study of oscillatory integrals throughout harmonic analysis and PDE theory, and is the first ingredient needed to attack the pointwise convergence questions taken up next in Convergence of Fourier Series.

Ask the tutor: How does Bessel's inequality change if we use complex exponentials $e^{inx}$ instead of $\cos nx,\sin nx$, and why does the complex form make Parseval's identity look even cleaner?

Think of Riemann integrable functions on $[-\pi,\pi]$ as vectors, with $\langle f,g\rangle = \int f g$ playing the role of the dot product. Two functions are "perpendicular" exactly when this integral vanishes — and remarkably, $\cos mx$ and $\sin nx$ are perpendicular to each other, and to all other trigonometric functions of different frequency, for a completely elementary reason: products of sines and cosines at different integer frequencies oscillate and their oscillations cancel exactly over a full period. This is the same phenomenon as an orthogonal basis $e_1, e_2, e_3$ in $\mathbb{R}^3$: to find the coordinate of a vector $v$ along $e_1$, you don't solve a system of equations, you just take the dot product $\langle v, e_1\rangle / \|e_1\|^2$. Fourier coefficients are exactly this projection formula applied to the "axes" $\cos nx, \sin nx$. Once you see $a_n, b_n$ as coordinates, the partial sum $S_N f$ is recognizable as the orthogonal projection of $f$ onto the finite-dimensional subspace spanned by frequencies $0$ through $N$ — and orthogonal projections are always the closest point in the subspace, which is precisely the least-squares optimality theorem below. Bessel's inequality is then just Pythagoras: the squared length of $f$ is at least the sum of the squared lengths of its projections onto mutually perpendicular axes.

Example 1 — Bessel's inequality for an asymmetric sawtooth-type function. Let $f(x) = x$ on $(-\pi,\pi]$, extended $2\pi$-periodically (the standard sawtooth). Since $f$ is odd, $a_n = 0$ for all $n\ge0$. Compute $b_n$ using integration by parts:

The second integral is $0$ for $n \ge 1$ (shown in the Lemma). The boundary term:

Since $\cos n\pi = (-1)^n$, we get $b_n = \frac1\pi \cdot \left(-\frac{2\pi(-1)^n}{n}\right) = \frac{-2(-1)^n}{n} = \frac{2(-1)^{n+1}}{n}$. So $b_1 = 2, b_2 = -1, b_3 = \frac23, b_4 = -\frac12, \dots$ Now compute the right side of Bessel's inequality:

Check Bessel's inequality at $N=3$: since $a_n \equiv 0$,

Indeed $5.4444 < 6.5797$, consistent with the theorem. Taking $N\to\infty$, Bessel's inequality asserts $\sum_{n=1}^\infty \frac{4}{n^2} \le \frac{2\pi^2}{3}$, i.e. $4\sum \frac{1}{n^2} \le \frac{2\pi^2}{3}$. Using the classical value $\sum_{n=1}^\infty \frac1{n^2} = \frac{\pi^2}{6}$ (a fact provable independently, e.g. via Parseval's theorem — the equality case of Bessel, treated alongside Convergence of Fourier Series), we get $4\cdot\frac{\pi^2}6 = \frac{2\pi^2}{3}$ exactly, matching the right side with equality — a preview of the fact that Bessel's inequality becomes an equality (Parseval's identity) for this particular $f$.

Example 2 — a non-odd, non-even function: least-squares optimal quadratic trigonometric approximation. Let $f(x) = x^2$ on $(-\pi,\pi]$ shifted so that it is not symmetric about the natural comparison point — instead take

periodically extended. This function is neither even nor odd, so both $a_n$ and $b_n$ are generically nonzero, giving a genuinely asymmetric test case. Coefficients from $x^2$ (even part): a standard computation (integration by parts twice) gives, for $n\ge1$,

so the contribution to $a_n$ from $x^2$ is $\frac1\pi\cdot \frac{4\pi(-1)^n}{n^2} = \frac{4(-1)^n}{n^2}$. Also $\int_{-\pi}^\pi x^2\,dx = \frac{2\pi^3}{3}$ so the $a_0$ contribution is $\frac{1}{\pi}\cdot\frac{2\pi^3}3 = \frac{2\pi^2}{3}$, giving constant term $a_0/2 = \pi^2/3$. Coefficients from $x$ (odd part, computed in Example 1): contributes $0$ to every $a_n$, and contributes $b_n = \frac{2(-1)^{n+1}}{n}$. Since Fourier coefficients are linear in $f$ (immediate from linearity of the integral), the coefficients of $g = x^2+x$ are the sums:

Let us be careful and just state them directly: $a_0 = \frac{2\pi^2}{3}$ (unchanged, since $x$ contributes $0$ to $a_0$ as it is odd), and for $n \ge 1$,

Take $N=2$. Then $a_1 = -4, a_2 = 1$; $b_1 = 2, b_2=-1$. The degree-$2$ Fourier partial sum is

By the Least-Squares Optimality Theorem, this specific trigonometric polynomial, among all trigonometric polynomials $c_0/2 + c_1\cos x + d_1 \sin x + c_2 \cos 2x + d_2 \sin 2x$, minimizes $\int_{-\pi}^\pi (g - T)^2\,dx$. No other choice of the five real parameters $(c_0,c_1,d_1,c_2,d_2)$ can do better — a claim that would be very hard to verify directly by calculus of five variables, but is immediate from the completed-square argument in the proof. Now check Bessel's inequality numerically. We need $\int_{-\pi}^\pi g(x)^2\,dx = \int_{-\pi}^\pi (x^2+x)^2\,dx = \int_{-\pi}^\pi (x^4 + 2x^3 + x^2)\,dx$. Since $x^3$ is odd, its integral over $(-\pi,\pi)$ vanishes. So

Numerically, $\pi \approx 3.14159265$, $\pi^3 \approx 31.00628$, $\pi^5 \approx 306.0197$. So $\frac{2\pi^5}{5}\approx 122.4079$ and $\frac{2\pi^3}{3}\approx 20.6709$, total $\approx 143.0788$. Dividing by $\pi$: $\frac1\pi\int g^2 \approx 45.5455$. Left side of Bessel at $N=2$:

Since $\pi^4 \approx 97.409$, $\frac{4\pi^4}{9}\approx 43.293$, divided by $2$ gives $\approx 21.6467$. Total left side $\approx 21.6467 + 20 + 2 = 43.6467$. Indeed $43.6467 < 45.5455$: Bessel's inequality holds, with a visible gap (the "energy" still uncaptured by frequencies $0,1,2$ is about $1.90$, matching exactly $\|g - S_2 g\|_2^2/\pi$ by the exact formula in the Optimality Theorem).