Taylor's Theorem in Several Variables

The second-order Taylor expansion is the algebraic core of the second-derivative test for classifying critical points in Critical Points and Extrema: at a critical point $a$ where $\nabla f(a) = 0$, the expansion collapses to $f(a+h) = f(a) + \tfrac12 h^T H_f(a) h + o(\|h\|^2)$, so the local behavior of $f$ near $a$ is governed entirely by the definiteness of The Hessian Matrix — positive definite gives a local minimum, negative definite a local maximum, indefinite a saddle. In numerical optimization, Newton's method for minimizing a function repeatedly builds the quadratic model $P_2$ at the current iterate and jumps to its minimizer, exploiting exactly this expansion; the remainder bound $O(\|h\|^3)$ explains the method's famous quadratic local convergence rate. In numerical analysis more broadly, finite-difference schemes for partial differential equations are derived and error-bounded using multivariable Taylor expansions with explicit remainder terms. In physics, expanding a potential energy function to second order near an equilibrium point produces the harmonic oscillator approximation, with the Hessian playing the role of a stiffness (or spring-constant) matrix whose eigenvalues give normal-mode frequencies. In statistics and machine learning, second-order (quadratic) approximations of loss functions underlie the Laplace approximation for Bayesian posteriors and quasi-Newton optimization algorithms such as BFGS.

:::faq[Can the theorem fail if $U$ is open but not convex, even if the specific segment $[a,a+h]$ stays inside $U$?] No — convexity of $U$ is never actually used in the proof. The only requirement is that the particular segment from $a$ to $a+h$ lies in $U$; the rest of $U$'s shape is irrelevant. So on a non-convex open set, Taylor's theorem still holds for pairs $(a,h)$ whose connecting segment happens to avoid the "missing" region, but may fail (or simply not apply) for pairs whose segment exits the domain, exactly as illustrated in the counterexample with the slit plane.

Ask the tutor: How does the Hessian's definiteness at a critical point actually determine whether it's a max, min, or saddle?

In one variable, Taylor's theorem says a smooth function looks, near $a$, like a polynomial in $(x-a)$: a constant term (the value), a linear term (the slope), a quadratic term (the curvature), and so on. In several variables the same idea holds, but "slope" becomes a direction-dependent number — the directional derivative $\nabla f(a)\cdot h$ — and "curvature" becomes a quadratic form $h^T H_f(a) h$ that can be positive in some directions and negative in others. The Hessian matrix is precisely the object that records this direction-dependent bending. The cleanest way to see why the formula must involve $\nabla f(a)\cdot h$ and $h^T H_f(a) h$ is to freeze the direction: fix $h$, and look at the single real function $g(t) = f(a+th)$ for $t$ near $0$. This is an ordinary one-variable function, so its own Taylor expansion at $t=0$ is

and the chain rule converts $g'(0)$ into $\nabla f(a)\cdot h$ and $g''(0)$ into $h^T H_f(a) h$. Setting $t=1$ recovers the multivariable expansion. This restriction-to-a-line trick is not just an analogy; it is the actual proof technique used below.

Example 1: A non-polynomial function at an asymmetric point. Let $f(x,y) = e^x \cos y$ (globally $C^\infty$ on all of $\mathbb{R}^2$, in particular $C^3$), and expand about the deliberately asymmetric base point $a = (1, \pi/3)$, i.e. compute $P_2(h)$ for $h=(h_1,h_2)$ near $0$. First compute the needed partials:

At $a=(1,\pi/3)$: $\cos(\pi/3) = \tfrac12$, $\sin(\pi/3) = \tfrac{\sqrt3}{2}$, so

So the gradient is $\nabla f(a) = \left(\tfrac{e}{2}, -\tfrac{e\sqrt3}{2}\right)$ and the Hessian is

By the theorem, writing $h=(h_1,h_2)$,

Simplify:

Numerical check. Take $h = (0.1, -0.2)$, i.e. approximate $f(1.1, \pi/3 - 0.2)$. Exact value: $x=1.1$, $y = \pi/3 - 0.2 \approx 1.04720 - 0.2 = 0.84720$. Compute $e^{1.1} \approx 3.00417$ and $\cos(0.84720) \approx 0.66235$. So $f(a+h) \approx 3.00417 \times 0.66235 \approx 1.99039$. Taylor estimate: plug $h_1 = 0.1$, $h_2=-0.2$ into $P_2(h)$ with $e/2 \approx 1.35914$:

So $P_2(h) \approx 1.35914 \times 1.46605 \approx 1.99277$. The two agree to about $0.0024$ absolute error, and the theorem guarantees this discrepancy is $O(\|h\|^3)$ — with $\|h\|_\infty = 0.2$, indeed $0.2^3 = 0.008$ is the right order of magnitude, comfortably bounding the observed error once the constant $\tfrac{n^3M_3}{6}$ is included.

Example 2: Verifying the remainder bound explicitly for a cubic. Let $f(x,y) = x^3 + 2x^2y - xy^2 + y^3$, expanded at $a = (2,-1)$ (again asymmetric, with mixed signs), evaluated at $h=(1,1)$ (so target point $(3,0)$). Since $f$ is a cubic polynomial, all fourth-order partials vanish, so the Lagrange remainder $R_2(h)$ should reproduce the exact cubic terms with no error left over (a good sanity check on the formula, since the theorem's remainder must exactly capture whatever the theorem's second-order part misses). Compute all needed derivatives:

All third partials are constant, so evaluating them at $a+\theta h$ for any $\theta$ gives the same values — meaning the remainder is exact for any choice of $\theta \in (0,1)$, consistent with the theorem (which only asserts existence of some $\theta$, but here every $\theta$ works because the third derivatives don't vary). At $a=(2,-1)$:

So

Remainder, using constant third partials $f_{xxx}=6,\,f_{xxy}=4,\,f_{xyy}=-2,\,f_{yyy}=6$ and multinomial coefficients $\binom{3}{3,0,0}=1,\binom{3}{2,1,0}=3,\binom{3}{1,2,0}=3,\binom{3}{0,3,0}=1$ for the pure/mixed terms in $h=(1,1)$:

Exact value: $f(3,0) = 27 + 2(9)(0) - 3(0) + 0 = 27$. And indeed $P_2(h) + R_2(h) = 24+3 = 27$. The identity is exact, confirming the formula's bookkeeping (multinomial coefficients, index collapsing) is correct.

:::counterexample[Why the segment $[a,a+h]$ must lie inside $U$] Drop the hypothesis that $[a,a+h] \subset U$, keeping $f \in C^3$ on its domain $U$. Let

(the plane with the non-negative $y$-axis removed), and let $f(x,y) = \arg(x+iy) \in (-\pi, \pi)$ be a continuous branch of the argument function on $U$ (well-defined and $C^\infty$ there, since removing a ray from the plane makes the multivalued argument single-valued and smooth). Take $a = (1,-1)$, so $f(a) = \arg(1-i) = -\pi/4$, and take $h = (-2, 2)$, so $a+h = (-1,1)$ and $f(a+h) = \arg(-1+i) = 3\pi/4$. The straight segment from $(1,-1)$ to $(-1,1)$ is the set $\{(t,-t): t\in[-1,1]\}$ — wait, more carefully, parametrize $a+th = (1-2t,\,-1+2t)$ for $t\in[0,1]$; at $t=1/2$ this is $(0,0)$, which lies on the boundary $x=0$ but is not in $U$ (since $y=0\ge 0$ is exactly the removed ray's endpoint), so the segment actually exits $U$ near $t=1/2$ (for $t$ slightly less than $1/2$, $x = 1-2t>0$ small and $y=-1+2t<0$ close to the boundary, but for $t$ slightly more than $1/2$, $x<0$ — the path crosses arbitrarily close to the removed ray at points with $x=0, y>0$ only in the limit; to make the violation unambiguous, perturb $h$ slightly). Sharper version: take $a = (1,-\epsilon)$ and $a+h = (-1,\epsilon)$ for small $\epsilon>0$, so the segment passes through $(0,0)\notin U$ exactly at $t=1/2$ regardless of $\epsilon$, hence $[a,a+h]\not\subset U$. As $\epsilon \to 0^+$, $f(a) = \arg(1-i\epsilon) \to 0^-$ while $f(a+h) = \arg(-1+i\epsilon) \to \pi^-$. Every partial derivative of $f$ (gradient, Hessian, third derivatives) stays uniformly bounded near $a$ as $\epsilon\to0$ (they depend smoothly on $a$, which stays near $(1,0)$, away from the branch cut), so the right-hand side $P_2(h) + R_2(h)$, if the theorem's formula were blindly applied using derivatives only at $a$, would predict $f(a+h) - f(a)$ to be a small, bounded quantity — yet the true difference $f(a+h)-f(a) \to \pi$ as $\epsilon\to 0$, a jump of order $\pi$, not small. The naive formula fails outright because the straight path from $a$ to $a+h$ leaves the domain of smoothness of $f$: there is no valid $\theta \in (0,1)$ with $a+\theta h \in U$ at the crossing point, so the Lagrange-form remainder cannot even be evaluated there, and no true equality holds. This shows the segment hypothesis is not a technicality: it is what allows the one-variable reduction $g(t)=f(a+th)$ to be defined and differentiable on all of $[0,1]$ in the first place.