The Arzelà-Ascoli Theorem

The Arzelà–Ascoli theorem is the standard mechanism for proving existence theorems where no explicit formula for the solution is available. In ordinary differential equations, Peano's existence theorem (for $y' = f(x,y)$ with only continuous $f$, no Lipschitz condition) is proved by constructing a sequence of approximate solutions (Euler polygons or Picard-type iterates), showing this sequence is uniformly bounded and equicontinuous via a bound on the derivatives, and then extracting a uniformly convergent subsequence whose limit is verified to solve the original equation. The same scheme underlies existence proofs in the calculus of variations (direct method: minimizing sequences are shown equicontinuous via bounds on an energy functional, then compactness produces a minimizer) and in the theory of optimal control. In complex analysis, Montel's theorem — that a locally bounded family of holomorphic functions on a domain is normal (every sequence has a subsequence converging locally uniformly) — is essentially Arzelà–Ascoli in disguise: local boundedness of holomorphic functions implies, via the Cauchy integral formula, local boundedness of all derivatives, which in turn implies equicontinuity, and the theorem applies directly. This is the engine behind the proof of the Riemann Mapping Theorem. In functional analysis and PDE theory, refinements of the same idea — the Rellich–Kondrachov compactness theorem and the Fréchet–Kolmogorov theorem — characterize compact subsets of Sobolev spaces and $L^p$ spaces respectively, using "equicontinuity in the mean" (uniform control of translates) instead of pointwise equicontinuity; these are indispensable in proving existence of weak solutions to elliptic and parabolic PDEs. In probability theory, an analogous tightness criterion (Prokhorov's theorem, and the Arzelà–Ascoli-flavored characterization of tightness for stochastic processes via Kolmogorov's continuity criterion) governs weak convergence of random processes, underlying the construction of Brownian motion as a limit of random walks. In machine learning and approximation theory, uniform boundedness and equicontinuity-type (Lipschitz) constraints on hypothesis classes are used to control generalization and to guarantee that empirical risk minimizers converge, connecting this 19th-century theorem to modern statistical learning theory. It is also the natural stepping stone toward the Stone-Weierstrass Theorem, which addresses a complementary question — not which families are compact, but which subalgebras of $C(K)$ are dense — and the two theorems together give a remarkably complete picture of the structure of $C(K)$.

:::quiz[Which single additional hypothesis, together with pointwise boundedness of $(f_n)$ on $[0,1]$, guarantees a uniformly convergent subsequence by Arzelà–Ascoli?]

• Each $f_n$ is differentiable
• The family is equicontinuous ✓
• Each $f_n$ is monotonic
• The family is uniformly bounded by a constant depending on $n$

:::quiz[Why does $f_n(x) = \sin(nx)$ on $[0, 2\pi]$ fail to have a uniformly convergent subsequence despite being uniformly bounded?]

• It is not a sequence of continuous functions
• $[0,2\pi]$ is not compact
• The family is not equicontinuous: oscillation frequency grows without bound ✓
• Uniform boundedness alone is never sufficient in any setting

Ask the tutor: Can you show me, step by step, how the Arzelà–Ascoli theorem is used inside the proof of Peano's existence theorem for ODEs?

Think first of ordinary Bolzano–Weierstrass: a bounded sequence of real numbers $(a_n) \subset [-M, M]$ always has a convergent subsequence, because $[-M,M]$ is compact. Now imagine trying the same trick with functions. Take the sequence $f_n(x) = \sin(nx)$ on $[0, 2\pi]$. Every $f_n$ satisfies $|f_n(x)| \le 1$, so the family is uniformly bounded — exactly the hypothesis that sufficed for numbers. Yet no subsequence of $(\sin(nx))$ converges uniformly, or even pointwise almost everywhere, to a continuous function: the oscillations get faster and faster, and any two functions $\sin(n_k x)$, $\sin(n_{k+1}x)$ in a candidate subsequence stay far apart in sup-norm no matter how far out you go. Boundedness alone was not the deep content of Bolzano–Weierstrass; it needs a companion. The missing ingredient is a uniform control on the rate of change — equicontinuity. It says: given any tolerance $\varepsilon$, there is a single $\delta$ that works for every function in the family simultaneously, guaranteeing that points within $\delta$ of each other have function values within $\varepsilon$. This is a much stronger requirement than each individual $f_n$ merely being continuous (which only guarantees a $\delta$ depending on both $\varepsilon$ and $f_n$). Equicontinuity rules out the $\sin(nx)$ example instantly: as $n \to \infty$ the functions become steeper and steeper near their zeros, so no single $\delta$ can control all of them. Once you have both hypotheses, the strategy for building a uniformly convergent subsequence is a two-stage extraction. First, restrict attention to a countable dense subset of $K$ (which exists because compact metric spaces are separable) and use a diagonal argument to find a subsequence converging at every one of these countably many points — this uses only boundedness and a diagonal trick à la Bolzano–Weierstrass, applied countably many times. Second, use equicontinuity to show that convergence at a dense set of points, for an equicontinuous family, automatically upgrades to uniform convergence on all of $K$. The two pictures below show the two hypotheses at work: uniform boundedness confines the graphs to a horizontal band, and equicontinuity confines the "steepness" so that a $\delta$-neighborhood in $x$ controls an $\varepsilon$-neighborhood in $y$, uniformly across the whole family.

We isolate two lemmas used in the proof.

Example 1 — A family that is bounded but not equicontinuous, examined function-by-function. Let $K = [0,1]$ and consider

First we check uniform boundedness. Differentiating (treating $n$ as fixed), $f_n'(x) = \dfrac{n(1 - n^2x^2)}{(1+n^2x^2)^2}$, which vanishes at $x = 1/n$, giving the maximum value

Since $f_n(0) = 0$ and $f_n(x) \ge 0$ on $[0,1]$ with this single interior critical point, $\sup_{x\in[0,1]} f_n(x) = 1/2$ for every $n \ge 1$; hence $|f_n(x)| \le 1/2$ for all $n,x$, so $\{f_n\}$ is uniformly bounded (with $M = 1/2$). Now examine equicontinuity. Compute

The two points $x=0$ and $x = 1/n$ satisfy $d(0, 1/n) = 1/n \to 0$, yet

for every $n$. So take $\varepsilon_0 = 1/4$: no matter how small $\delta > 0$ is, choosing $n > 1/\delta$ gives two points within $\delta$ of each other (namely $0$ and $1/n$) where $f_n$ differs by $1/2 \ge \varepsilon_0$. Hence $\{f_n\}$ is not equicontinuous — the bump of height $1/2$ simply migrates toward $x=0$ and narrows as $n \to \infty$, so no single $\delta$ can control all $f_n$ simultaneously near $0$. Consistent with the theorem's necessity direction, this family indeed fails to have a uniformly convergent subsequence: pointwise, $f_n(x) \to 0$ for every fixed $x \in [0,1]$ (check: for $x = 0$ trivially; for $x > 0$ fixed, $f_n(x) = \frac{nx}{1+n^2x^2} \sim \frac{1}{nx} \to 0$). So the only possible uniform limit of any subsequence would be the zero function. But $\rho(f_n, 0) = \sup_x f_n(x) = 1/2$ for every $n$, so $f_n \not\to 0$ uniformly, and no subsequence converges to $0$ uniformly either (the sup-distance to $0$ is $1/2$ along the whole sequence, not just a subsequence). Hence no subsequence converges uniformly at all — exactly as the theorem predicts once equicontinuity fails, even though boundedness held.

Example 2 — Verifying the hypotheses directly from a differential inequality, and extracting the limit. Let $K = [0,1]$ and let $\mathcal F$ be the family of all solutions $y$ on $[0,1]$ of the initial value problems

one solution $y_c$ for each $c \in [-1,1]$ (existence and uniqueness of each such solution is guaranteed by the Picard–Lindelöf theorem, since the right side is smooth and globally Lipschitz in $y$ on bounded sets — we take this as given input data for the family). We verify $\mathcal F = \{y_c : c \in [-1,1]\}$ satisfies the hypotheses of Arzelà–Ascoli and extract a convergent subsequence for a specific sequence $c_n$. Uniform boundedness. Since $|\sin(xy_c(x))| \le 1$ and $|c| \le 1$, we get $|y_c'(x)| \le 1 + 1 = 2$ for all $x \in [0,1]$ and all $c \in [-1,1]$. Hence, by the Mean Value Theorem (or by integrating), for $x \in [0,1]$,

So $\mathcal F$ is uniformly bounded by $M = 3$. Equicontinuity. From $|y_c'(x)| \le 2$ for all $c, x$, the Mean Value Theorem gives, for any $x, x' \in [0,1]$ and any $c \in [-1,1]$,

This is a single Lipschitz bound valid for every function in the family (uniform in $c$), so given $\varepsilon > 0$, choosing $\delta = \varepsilon/2$ gives $|x-x'| < \delta \implies |y_c(x) - y_c(x')| < \varepsilon$ for every $c \in [-1,1]$ simultaneously. Hence $\mathcal F$ is equicontinuous. By the Arzelà–Ascoli theorem, $\mathcal F$ is relatively compact in $C[0,1]$: every sequence of solutions has a uniformly convergent subsequence. Concretely, take $c_n = (-1)^n \cdot \frac{n}{n+1}$, so that $c_n$ oscillates and does not itself converge (it has two cluster points, $1$ and $-1$). The sequence $y_{c_n}$ need not converge as a whole, but Arzelà–Ascoli guarantees a subsequence does converge uniformly — for instance the subsequence with even $n$ has $c_{2m} = \frac{2m}{2m+1} \to 1$, and by continuous dependence of solutions on the initial condition $c$ (itself provable using the same equicontinuity/Gronwall-type estimate), $y_{c_{2m}} \to y_1$ uniformly on $[0,1]$, where $y_1$ is the solution with $c=1$. This illustrates the theorem's typical use: it guarantees the existence of a convergent subsequence with a specific limit identifiable by independent means (here, continuous dependence on parameters), without needing an explicit closed-form for $y_c$.