The Gradient and Directional Derivatives

Stand on a hillside modeled by a height function f(x,y)f(x,y). If you take a step in some direction, how fast does your altitude change? This question has infinitely many answers. There is one for each direction you might walk. Yet all of them are governed by a single vector attached to your location: the gradient. Understanding this vector converts a seemingly infinite family of rates of change into one compact piece of data. This matters because most of the interesting objects in science are scalar-valued functions of several variables. These include temperature fields, potential energy, pressure, probability densities, and loss functions in machine learning. The question "which way does it increase fastest, and how fast?" is asked constantly. The gradient answers both parts of that question simultaneously. It does so by packaging every directional derivative into one linear functional. The big picture is this: differentiability is already established as the existence of a good linear approximation to ff near a point. This is exactly the hypothesis needed to guarantee something important. It guarantees that all directional derivatives exist, vary predictably with direction, and are computed by a single dot product with the gradient. Without differentiability, directional derivatives can exist in every direction. Yet they can fail to combine into anything so simple. We exhibit this phenomenon explicitly. This article builds the directional derivative from the total derivative. It proves the gradient formula and the steepest-ascent theorem. It establishes that gradients are orthogonal to level sets. It also works through the subtleties. These include the classic counterexample of a function with all directional derivatives but no gradient behavior at all.

Key takeaways
  • The directional derivative Duf(p)D_{\mathbf{u}}f(\mathbf{p}) measures the instantaneous rate of change of ff at p\mathbf{p} in direction u\mathbf{u}.
  • If ff is differentiable at p\mathbf{p}, then Duf(p)=∇f(p)⋅uD_{\mathbf{u}}f(\mathbf{p}) = \nabla f(\mathbf{p}) \cdot \mathbf{u} for every unit vector u\mathbf{u}.
  • The gradient ∇f(p)\nabla f(\mathbf{p}) points in the direction of steepest ascent, and ∥∇f(p)∥\|\nabla f(\mathbf{p})\| is that maximal rate.
  • The gradient is orthogonal to the level set of ff through p\mathbf{p}, wherever ∇f(p)≠0\nabla f(\mathbf{p}) \neq \mathbf{0}.
  • Existence of all directional derivatives at a point does not imply differentiability there, so the dot-product formula can fail.
  • Key idea: differentiability turns the infinite family of directional derivatives into one linear functional, the gradient, via Duf=∇f⋅uD_{\mathbf u}f = \nabla f \cdot \mathbf u.

Where this lives today

The gradient is the computational engine of gradient descent and its variants (stochastic gradient descent, Adam, momentum methods), which train nearly every modern machine learning model by repeatedly stepping in the direction −∇f-\nabla f to decrease a loss function ff. In physics, the gradient of a potential energy function gives (up to sign) the force field, F=−∇U\mathbf F = -\nabla U, linking this purely calculus-based construction to Newtonian mechanics and to the theory of conservative vector fields. In thermodynamics and fluid dynamics, gradients of temperature or pressure fields drive heat flow (Fourier's law, q=−k∇T\mathbf q = -k\nabla T) and fluid motion. In computer graphics and image processing, image gradients detect edges by identifying directions and magnitudes of rapid intensity change. In optimization theory more broadly, the gradient underlies first-order necessary conditions for extrema, feeding directly into constrained optimization via Lagrange Multipliers, where the orthogonality of ∇f\nabla f to level sets becomes the geometric heart of the multiplier condition. In differential geometry, the gradient generalizes to Riemannian manifolds as the vector field dual to dfdf under the metric, extending every idea in this article beyond flat Euclidean space.

Ask the tutor: How does the gradient formula change when ff is defined on a curved surface instead of flat Euclidean space?

DefinitionDirectional derivative and gradient

Let f:U→Rf : U \to \mathbb{R} be defined on an open set U⊆RnU \subseteq \mathbb{R}^n, let p∈U\mathbf{p} \in U, and let u∈Rn\mathbf{u} \in \mathbb{R}^n be a unit vector (∥u∥=1\|\mathbf{u}\| = 1). The directional derivative of ff at p\mathbf{p} in the direction u\mathbf{u} is

Duf(p)  =  lim⁡h→0f(p+hu)−f(p)h,D_{\mathbf{u}}f(\mathbf{p}) \;=\; \lim_{h \to 0} \frac{f(\mathbf{p} + h\mathbf{u}) - f(\mathbf{p})}{h},

provided this limit exists. (Here hh ranges over real numbers near 00, positive or negative; since UU is open, p+hu∈U\mathbf{p}+h\mathbf u \in U for all sufficiently small ∣h∣|h|.) If ff is differentiable at p\mathbf{p}, all partial derivatives ∂f/∂xi(p)\partial f/\partial x_i(\mathbf{p}) exist, and we define the gradient of ff at p\mathbf{p} as the vector of partial derivatives

∇f(p)  =  (∂f∂x1(p),  ∂f∂x2(p),  …,  ∂f∂xn(p)).\nabla f(\mathbf{p}) \;=\; \left( \frac{\partial f}{\partial x_1}(\mathbf{p}), \; \frac{\partial f}{\partial x_2}(\mathbf{p}), \; \dots, \; \frac{\partial f}{\partial x_n}(\mathbf{p}) \right).

Recall (from Differentiability and the Total Derivative) that ff is differentiable at p\mathbf{p} if there exists a linear map Df(p):Rn→RDf(\mathbf{p}) : \mathbb{R}^n \to \mathbb{R} such that

lim⁡h→0f(p+h)−f(p)−Df(p)(h)∥h∥=0.\lim_{\mathbf{h} \to \mathbf{0}} \frac{f(\mathbf{p}+\mathbf{h}) - f(\mathbf{p}) - Df(\mathbf{p})(\mathbf{h})}{\|\mathbf{h}\|} = 0.

When this holds, Df(p)(h)=∇f(p)⋅hDf(\mathbf{p})(\mathbf{h}) = \nabla f(\mathbf{p}) \cdot \mathbf{h} for all h∈Rn\mathbf{h} \in \mathbb{R}^n.

Intuition

Think of f(x,y)f(x,y) as elevation on a topographic map. Fix a point p\mathbf{p} and imagine standing there facing some compass direction u\mathbf{u}. The directional derivative Duf(p)D_{\mathbf{u}}f(\mathbf{p}) is the slope you feel under your feet as you begin walking in that direction — positive if you're climbing, negative if descending, zero if you're momentarily walking level. If ff is merely differentiable, the graph of ff near p\mathbf{p} looks, at high magnification, like a plane (the tangent plane). Walking on a plane in any direction produces a slope that depends on direction in the simplest possible way: linearly, via a dot product with a fixed "tilt vector." That tilt vector is exactly ∇f(p)\nabla f(\mathbf{p}). Since Duf(p)=∇f(p)⋅u=∥∇f(p)∥cos⁡θD_{\mathbf u}f(\mathbf p) = \nabla f(\mathbf p)\cdot \mathbf u = \|\nabla f(\mathbf p)\|\cos\theta, where θ\theta is the angle between u\mathbf u and the gradient, the slope is maximized when θ=0\theta = 0 — that is, when you walk along the gradient — and it is maximized precisely at the value ∥∇f(p)∥\|\nabla f(\mathbf p)\|. Walking perpendicular to the gradient (θ=π/2\theta = \pi/2) keeps you level: this is the direction tangent to the level curve through p\mathbf p.

Figure
The gradient as steepest ascent, orthogonal to the level curve
f = 3 f = 2 f = 1 p ∇f(p) tangent to level curve ∇f(p) is perpendicular to the level curve; it points toward increasing f

Deriving the formula

The key move is to reduce the directional derivative to ordinary single-variable differentiation of the composite function t↦f(p+tu)t \mapsto f(\mathbf p + t\mathbf u), and to exploit the definition of differentiability directly (not the chain rule as a black box, since the chain rule is normally proved from this kind of argument).

TheoremDirectional derivative via the gradient

Let U⊆RnU \subseteq \mathbb{R}^n be open, let f:U→Rf: U \to \mathbb{R}, let p∈U\mathbf{p} \in U, and suppose ff is differentiable at p\mathbf{p}. Then for every unit vector u∈Rn\mathbf{u} \in \mathbb{R}^n, the directional derivative Duf(p)D_{\mathbf{u}}f(\mathbf{p}) exists and

Duf(p)=∇f(p)⋅u.D_{\mathbf{u}}f(\mathbf{p}) = \nabla f(\mathbf{p}) \cdot \mathbf{u}.

Consequently: 1. (Steepest ascent) If ∇f(p)≠0\nabla f(\mathbf p) \neq \mathbf 0, then Duf(p)D_{\mathbf u}f(\mathbf p) is maximized over unit vectors u\mathbf u exactly at u∗=∇f(p)/∥∇f(p)∥\mathbf u^* = \nabla f(\mathbf p)/\|\nabla f(\mathbf p)\|, with maximum value ∥∇f(p)∥\|\nabla f(\mathbf p)\|; it is minimized at −u∗-\mathbf u^*, with minimum value −∥∇f(p)∥-\|\nabla f(\mathbf p)\|. 2. (Level-set orthogonality) If γ:(−ε,ε)→U\gamma:(-\varepsilon,\varepsilon)\to U is differentiable with γ(0)=p\gamma(0)=\mathbf p and f(γ(t))=f(p)f(\gamma(t)) = f(\mathbf p) for all tt (i.e. γ\gamma traces the level set through p\mathbf p), then

∇f(p)⋅γ′(0)=0.\nabla f(\mathbf{p}) \cdot \gamma'(0) = 0.
Proof

Step 1: Reduce to a one-variable limit using differentiability directly. Fix a unit vector u\mathbf u. Since UU is open and p∈U\mathbf p \in U, there is δ>0\delta > 0 such that p+hu∈U\mathbf p + h\mathbf u \in U for all ∣h∣<δ|h| < \delta. By the hypothesis that ff is differentiable at p\mathbf p, there is a linear map L=Df(p):Rn→RL = Df(\mathbf p): \mathbb R^n \to \mathbb R, given by L(v)=∇f(p)⋅vL(\mathbf v) = \nabla f(\mathbf p)\cdot \mathbf v, such that

f(p+h)−f(p)=L(h)+R(h),where lim⁡h→0R(h)∥h∥=0.(∗)f(\mathbf p + \mathbf h) - f(\mathbf p) = L(\mathbf h) + R(\mathbf h), \qquad \text{where } \lim_{\mathbf h \to 0} \frac{R(\mathbf h)}{\|\mathbf h\|} = 0. \tag{$\ast$}

(Here R(h):=f(p+h)−f(p)−L(h)R(\mathbf h) := f(\mathbf p+\mathbf h)-f(\mathbf p) - L(\mathbf h) is simply defined by this equation; the content of differentiability is the limit condition on RR.)

Step 2: Substitute h=hu\mathbf h = h\mathbf u. For real hh with 0<∣h∣<δ0<|h|<\delta, set h=hu\mathbf h = h\mathbf u in (∗)(\ast). Since LL is linear and ∥u∥=1\|\mathbf u\|=1,

f(p+hu)−f(p)=L(hu)+R(hu)=h L(u)+R(hu)=h(∇f(p)⋅u)+R(hu).f(\mathbf p + h\mathbf u) - f(\mathbf p) = L(h\mathbf u) + R(h\mathbf u) = h\, L(\mathbf u) + R(h\mathbf u) = h \big(\nabla f(\mathbf p)\cdot \mathbf u\big) + R(h\mathbf u).

Divide by h≠0h \neq 0:

f(p+hu)−f(p)h=∇f(p)⋅u+R(hu)h.\frac{f(\mathbf p + h\mathbf u) - f(\mathbf p)}{h} = \nabla f(\mathbf p)\cdot \mathbf u + \frac{R(h\mathbf u)}{h}.

Step 3: Show the remainder term vanishes as h→0h \to 0. For h≠0h \neq 0, ∥hu∥=∣h∣⋅∥u∥=∣h∣\|h\mathbf u\| = |h|\cdot\|\mathbf u\| = |h|, so

∣R(hu)h∣=∣R(hu)∣∣h∣=∣R(hu)∣∥hu∥.\left|\frac{R(h\mathbf u)}{h}\right| = \frac{|R(h\mathbf u)|}{|h|} = \frac{|R(h\mathbf u)|}{\|h\mathbf u\|}.

As h→0h \to 0 (with h≠0h \neq 0), hu→0h\mathbf u \to \mathbf 0, so by the limit condition in (∗)(\ast), R(hu)∥hu∥→0\dfrac{R(h\mathbf u)}{\|h\mathbf u\|} \to 0. Hence R(hu)h→0\dfrac{R(h\mathbf u)}{h} \to 0 as h→0h \to 0.

Step 4: Conclude the limit exists and equals ∇f(p)⋅u\nabla f(\mathbf p)\cdot\mathbf u. Taking h→0h \to 0 in Step 2's identity and using Step 3,

Duf(p)=lim⁡h→0f(p+hu)−f(p)h=∇f(p)⋅u+0=∇f(p)⋅u.D_{\mathbf u}f(\mathbf p) = \lim_{h\to 0} \frac{f(\mathbf p+h\mathbf u)-f(\mathbf p)}{h} = \nabla f(\mathbf p)\cdot \mathbf u + 0 = \nabla f(\mathbf p)\cdot \mathbf u.

This proves the main formula. In particular, taking u=ei\mathbf u = \mathbf e_i recovers Deif(p)=∂f/∂xi(p)D_{\mathbf e_i}f(\mathbf p) = \partial f/\partial x_i(\mathbf p), consistent with the definition of ∇f\nabla f.

Step 5 (Part 1 — steepest ascent): maximize the dot product. Suppose ∇f(p)≠0\nabla f(\mathbf p) \neq \mathbf 0. By the Cauchy–Schwarz inequality, for any unit vector u\mathbf u,

∇f(p)⋅u  ≤  ∥∇f(p)∥ ∥u∥=∥∇f(p)∥,\nabla f(\mathbf p)\cdot \mathbf u \;\le\; \|\nabla f(\mathbf p)\|\, \|\mathbf u\| = \|\nabla f(\mathbf p)\|,

with equality if and only if u\mathbf u is a nonnegative scalar multiple of ∇f(p)\nabla f(\mathbf p); since u\mathbf u is a unit vector this forces u=∇f(p)/∥∇f(p)∥=:u∗\mathbf u = \nabla f(\mathbf p)/\|\nabla f(\mathbf p)\| =: \mathbf u^*. (Equality in Cauchy–Schwarz a⋅b≤∥a∥∥b∥\mathbf a \cdot \mathbf b \le \|\mathbf a\|\|\mathbf b\| holds iff a,b\mathbf a, \mathbf b are linearly dependent with a nonnegative proportionality constant, a standard fact proved by expanding ∥a∥2∥b∥2−(a⋅b)2≥0\|\mathbf a\|^2\|\mathbf b\|^2 - (\mathbf a\cdot \mathbf b)^2 \ge 0 via the discriminant of t↦∥a−tb∥2t \mapsto \|\mathbf a - t\mathbf b\|^2.) So Duf(p)=∇f(p)⋅uD_{\mathbf u}f(\mathbf p) = \nabla f(\mathbf p)\cdot \mathbf u is uniquely maximized at u=u∗\mathbf u = \mathbf u^*, and the maximum value is ∇f(p)⋅u∗=∥∇f(p)∥2/∥∇f(p)∥=∥∇f(p)∥\nabla f(\mathbf p)\cdot \mathbf u^* = \|\nabla f(\mathbf p)\|^2/\|\nabla f(\mathbf p)\| = \|\nabla f(\mathbf p)\|. Replacing u\mathbf u by −u-\mathbf u everywhere (which is again a unit vector) shows D−uf(p)=−Duf(p)D_{-\mathbf u}f(\mathbf p) = -D_{\mathbf u}f(\mathbf p), so the minimum of Duf(p)D_{\mathbf u}f(\mathbf p) over unit vectors is −∥∇f(p)∥-\|\nabla f(\mathbf p)\|, attained uniquely at u=−u∗\mathbf u = -\mathbf u^*.

Step 6 (Part 2 — orthogonality to level sets): differentiate the constant composite. Let γ\gamma be as stated: differentiable at 00, γ(0)=p\gamma(0) = \mathbf p, and g(t):=f(γ(t))=f(p)g(t) := f(\gamma(t)) = f(\mathbf p) identically for tt near 00. Since gg is constant, g′(0)=0g'(0) = 0. We compute g′(0)g'(0) directly from differentiability of ff at p\mathbf p, exactly as in Steps 1–4 but with the path γ\gamma instead of the straight ray p+hu\mathbf p + h\mathbf u. Write h(t):=γ(t)−p\mathbf h(t) := \gamma(t) - \mathbf p, so h(t)→0\mathbf h(t) \to \mathbf 0 as t→0t \to 0 and h(t)/t→γ′(0)\mathbf h(t)/t \to \gamma'(0) by definition of γ′(0)\gamma'(0) (in particular h(t)≠0\mathbf h(t) \ne \mathbf 0 for small t≠0t\ne0 unless γ′(0)=0\gamma'(0) = \mathbf 0, in which case the conclusion ∇f(p)⋅γ′(0)=0\nabla f(\mathbf p)\cdot \gamma'(0) = 0 is trivial). By (∗)(\ast) applied with h=h(t)\mathbf h = \mathbf h(t),

g(t)−g(0)=f(p+h(t))−f(p)=∇f(p)⋅h(t)+R(h(t)).g(t) - g(0) = f(\mathbf p + \mathbf h(t)) - f(\mathbf p) = \nabla f(\mathbf p)\cdot \mathbf h(t) + R(\mathbf h(t)).

Divide by t≠0t \neq 0:

g(t)−g(0)t=∇f(p)⋅h(t)t+R(h(t))t.\frac{g(t)-g(0)}{t} = \nabla f(\mathbf p) \cdot \frac{\mathbf h(t)}{t} + \frac{R(\mathbf h(t))}{t}.

As t→0t \to 0: the left side →g′(0)=0\to g'(0) = 0 since gg is constant; the first term on the right →∇f(p)⋅γ′(0)\to \nabla f(\mathbf p)\cdot \gamma'(0) by linearity of the dot product and h(t)/t→γ′(0)\mathbf h(t)/t \to \gamma'(0); and the second term →0\to 0 by the same argument as Step 3 (writing R(h(t))/t=R(h(t))∥h(t)∥⋅∥h(t)∥tR(\mathbf h(t))/t = \frac{R(\mathbf h(t))}{\|\mathbf h(t)\|}\cdot\frac{\|\mathbf h(t)\|}{t}, where the first factor →0\to 0 by differentiability of ff and the second factor →∥γ′(0)∥\to \|\gamma'(0)\|, a bounded quantity, when γ′(0)≠0\gamma'(0)\ne \mathbf 0; if γ′(0)=0\gamma'(0) = \mathbf 0 the whole displayed identity gives 0=0+lim⁡R(h(t))/t0 = 0 + \lim R(\mathbf h(t))/t directly). Hence

0=∇f(p)⋅γ′(0),0 = \nabla f(\mathbf p)\cdot \gamma'(0),

as claimed. ■\blacksquare

∎

This proof technique — reducing a multivariable limit to a one-variable limit along a straight line or curve, then controlling the error term using the defining property of differentiability — is the standard method for deriving directional-derivative and chain-rule identities; it is not circular because it invokes only the definition of differentiability, never a later theorem such as the chain rule itself.

CorollaryGradient vanishes at interior extrema — a first look ahead

If ff is differentiable at p\mathbf p and p\mathbf p is a local maximum or local minimum of ff (relative to UU), then ∇f(p)=0\nabla f(\mathbf p) = \mathbf 0.

Proof

Suppose ∇f(p)≠0\nabla f(\mathbf p) \neq \mathbf 0. By Part 1 of the theorem above, taking u∗=∇f(p)/∥∇f(p)∥\mathbf u^* = \nabla f(\mathbf p)/\|\nabla f(\mathbf p)\| gives Du∗f(p)=∥∇f(p)∥>0D_{\mathbf u^*}f(\mathbf p) = \|\nabla f(\mathbf p)\| > 0. By definition of the directional derivative as a limit,

lim⁡h→0+f(p+hu∗)−f(p)h=∥∇f(p)∥>0,\lim_{h \to 0^+} \frac{f(\mathbf p + h\mathbf u^*) - f(\mathbf p)}{h} = \|\nabla f(\mathbf p)\| > 0,

so for all sufficiently small h>0h > 0, f(p+hu∗)>f(p)f(\mathbf p + h\mathbf u^*) > f(\mathbf p), contradicting that p\mathbf p is a local maximum. Similarly, using h→0−h \to 0^- or the direction −u∗-\mathbf u^*, one finds points arbitrarily close to p\mathbf p with ff-value below f(p)f(\mathbf p), contradicting a local minimum. Hence ∇f(p)=0\nabla f(\mathbf p) = \mathbf 0. ■\blacksquare (This is a preview of the first-order condition used systematically in Critical Points and Extrema and in Lagrange Multipliers.)

∎

Worked examples

Example 1 — steepest ascent for a non-symmetric quadratic. Let f(x,y)=3x2−xy+2y3f(x,y) = 3x^2 - xy + 2y^3 and let p=(1,−1)\mathbf p = (1, -1). We compute the directional derivative of ff at p\mathbf p in the direction of v=(3,4)\mathbf v = (3, 4) (not a unit vector), and find the direction and rate of steepest ascent. Gradient. Compute partials:

∂f∂x=6x−y,∂f∂y=−x+6y2.\frac{\partial f}{\partial x} = 6x - y, \qquad \frac{\partial f}{\partial y} = -x + 6y^2.

At p=(1,−1)\mathbf p = (1,-1): ∂f/∂x=6(1)−(−1)=7\partial f/\partial x = 6(1) - (-1) = 7, and ∂f/∂y=−1+6(1)=5\partial f/\partial y = -1 + 6(1) = 5. So

∇f(1,−1)=(7,5).\nabla f(1,-1) = (7, 5).

Since ff is a polynomial, it is differentiable everywhere (its partials exist and are continuous, which — by the standard sufficient condition built on the total-derivative definition — implies differentiability), so the Theorem applies. Directional derivative in the direction of v=(3,4)\mathbf v = (3,4). First normalize: ∥v∥=9+16=5\|\mathbf v\| = \sqrt{9+16} = 5, so u=(3/5,4/5)\mathbf u = (3/5, 4/5). Then

Duf(1,−1)=∇f(1,−1)⋅u=(7,5)⋅(35,45)=215+205=415=8.2.D_{\mathbf u}f(1,-1) = \nabla f(1,-1)\cdot \mathbf u = (7,5)\cdot\left(\tfrac{3}{5}, \tfrac{4}{5}\right) = \frac{21}{5} + \frac{20}{5} = \frac{41}{5} = 8.2.

So walking from (1,−1)(1,-1) in the direction (3,4)(3,4), ff increases at instantaneous rate 8.28.2 per unit distance. Steepest ascent. The direction of steepest ascent is u∗=∇f(1,−1)/∥∇f(1,−1)∥=(7,5)/74\mathbf u^* = \nabla f(1,-1)/\|\nabla f(1,-1)\| = (7,5)/\sqrt{74}, and the maximal rate of increase is ∥∇f(1,−1)∥=72+52=74≈8.602\|\nabla f(1,-1)\| = \sqrt{7^2+5^2} = \sqrt{74} \approx 8.602. Sanity check: the direction (3,4)/5(3,4)/5 makes an angle θ\theta with (7,5)(7,5) satisfying cos⁡θ=41/574=8.28.602≈0.9533\cos\theta = \frac{41/5}{\sqrt{74}} = \frac{8.2}{8.602} \approx 0.9533, so θ≈17.5∘\theta \approx 17.5^\circ — a modest deviation from the steepest direction, consistent with 8.28.2 being close to, but strictly less than, the maximum 8.6028.602.

Example 2 — level-curve orthogonality with an explicit parametrization. Let f(x,y)=x2y+y2f(x,y) = x^2 y + y^2, and consider the level curve f(x,y)=3f(x,y) = 3 through the point p=(1,2−2)\mathbf p = (1,\sqrt 2 - 2). We check numerically that ∇f(p)\nabla f(\mathbf p) is orthogonal to this curve, using an explicit local parametrization solved from the level-set equation, rather than assuming its existence. First verify p\mathbf p lies on the curve: f(1,2−2)=1⋅(2−2)+(2−2)2f(1, \sqrt2-2) = 1\cdot(\sqrt2-2) + (\sqrt2-2)^2. Compute (2−2)2=2−42+4=6−42(\sqrt2-2)^2 = 2 - 4\sqrt2 + 4 = 6-4\sqrt2. So f(p)=(2−2)+(6−42)=4−32≈4−4.2426=−0.2426f(\mathbf p) = (\sqrt2 - 2) + (6-4\sqrt2) = 4 - 3\sqrt2 \approx 4 - 4.2426 = -0.2426. That is not 33 — let's instead choose a genuinely clean point on the level curve f=3f=3: take x=1x=1, and solve y2+y−3=0y^2 + y - 3 = 0, giving y=−1+132≈−1+3.60562≈1.3028y = \frac{-1+\sqrt{13}}{2} \approx \frac{-1+3.6056}{2} \approx 1.3028. Set p=(1,y0)\mathbf p = (1, y_0) with y0=−1+132y_0 = \frac{-1+\sqrt{13}}{2}, so f(p)=3f(\mathbf p) = 3 exactly by construction. Gradient at p\mathbf p. ∂f/∂x=2xy\partial f/\partial x = 2xy, ∂f/∂y=x2+2y\partial f/\partial y = x^2 + 2y. At (1,y0)(1, y_0):

∇f(p)=(2y0,  1+2y0).\nabla f(\mathbf p) = (2y_0,\; 1 + 2y_0).

Numerically, 2y0≈2.60562y_0 \approx 2.6056 and 1+2y0≈3.60561+2y_0 \approx 3.6056, so ∇f(p)≈(2.6056, 3.6056)\nabla f(\mathbf p) \approx (2.6056,\, 3.6056). Local parametrization of the level curve. Since ∂f/∂y(p)=1+2y0≈3.6056≠0\partial f/\partial y(\mathbf p) = 1+2y_0 \approx 3.6056 \neq 0, the Implicit Function Theorem (built on differentiability, as established elsewhere) guarantees the level set f(x,y)=3f(x,y)=3 is locally the graph y=φ(x)y = \varphi(x) near p\mathbf p, with φ(1)=y0\varphi(1) = y_0. Implicit differentiation of x2y+y2=3x^2 y + y^2 = 3 gives

2xy+x2y′+2yy′=0⟹y′=−2xyx2+2y.2xy + x^2 y' + 2y y' = 0 \quad\Longrightarrow\quad y' = \frac{-2xy}{x^2+2y}.

At x=1,y=y0x=1, y=y_0: y′(1)=−2y01+2y0≈−2.60563.6056≈−0.7226y'(1) = \dfrac{-2y_0}{1+2y_0} \approx \dfrac{-2.6056}{3.6056} \approx -0.7226. A tangent vector to the curve at p\mathbf p is γ′(0)=(1,y′(1))≈(1,−0.7226)\gamma'(0) = (1, y'(1)) \approx (1, -0.7226) (parametrizing by xx itself). Check orthogonality.

∇f(p)⋅γ′(0)≈(2.6056)(1)+(3.6056)(−0.7226)≈2.6056−2.6058≈−0.0002≈0,\nabla f(\mathbf p) \cdot \gamma'(0) \approx (2.6056)(1) + (3.6056)(-0.7226) \approx 2.6056 - 2.6058 \approx -0.0002 \approx 0,

confirming (up to rounding) ∇f(p)⋅γ′(0)=0\nabla f(\mathbf p) \cdot \gamma'(0) = 0, exactly as the Theorem's Part 2 predicts. (The tiny residual is rounding error from truncating 13\sqrt{13}; algebraically the identity 2xy⋅1+(x2+2y)⋅(−2xyx2+2y)=2xy−2xy=02xy\cdot 1 + (x^2+2y)\cdot\left(\frac{-2xy}{x^2+2y}\right) = 2xy - 2xy = 0 is exact.)

CounterexampleExistence of all directional derivatives does not imply the gradient formula

The Theorem requires differentiability of ff at p\mathbf p, not merely the existence of directional derivatives in every direction. Weakening this hypothesis breaks the conclusion. Define f:R2→Rf : \mathbb R^2 \to \mathbb R by

f(x,y)={x2yx4+y2(x,y)≠(0,0)0(x,y)=(0,0).f(x,y) = \begin{cases} \dfrac{x^2 y}{x^4+y^2} & (x,y) \neq (0,0) \\ 0 & (x,y) = (0,0). \end{cases}

Claim: every directional derivative of ff at the origin exists, and is given by an explicit (non-linear) formula in the direction, so the map u↦Duf(0,0)\mathbf u \mapsto D_{\mathbf u}f(0,0) is not of the form u↦c⋅u\mathbf u \mapsto \mathbf c \cdot \mathbf u for any fixed vector c\mathbf c — in particular it is not linear, so no "gradient dot product" formula can hold. Let u=(cos⁡θ,sin⁡θ)\mathbf u = (\cos\theta, \sin\theta) be a unit vector. Case sin⁡θ≠0\sin\theta \neq 0. For small h≠0h \neq 0,

f(hcos⁡θ,hsin⁡θ)=h2cos⁡2θ⋅hsin⁡θh4cos⁡4θ+h2sin⁡2θ=h3cos⁡2θsin⁡θh2(h2cos⁡4θ+sin⁡2θ)=hcos⁡2θsin⁡θh2cos⁡4θ+sin⁡2θ.f(h\cos\theta, h\sin\theta) = \frac{h^2\cos^2\theta \cdot h\sin\theta}{h^4\cos^4\theta + h^2\sin^2\theta} = \frac{h^3 \cos^2\theta \sin\theta}{h^2(h^2\cos^4\theta + \sin^2\theta)} = \frac{h\cos^2\theta\sin\theta}{h^2\cos^4\theta+\sin^2\theta}.

So

Duf(0,0)=lim⁡h→0f(hu)−0h=lim⁡h→0cos⁡2θsin⁡θh2cos⁡4θ+sin⁡2θ=cos⁡2θsin⁡θsin⁡2θ=cos⁡2θsin⁡θ,D_{\mathbf u}f(0,0) = \lim_{h\to 0}\frac{f(h\mathbf u)-0}{h} = \lim_{h\to 0} \frac{\cos^2\theta \sin\theta}{h^2\cos^4\theta+\sin^2\theta} = \frac{\cos^2\theta\sin\theta}{\sin^2\theta} = \frac{\cos^2\theta}{\sin\theta},

which exists and is finite for every θ\theta with sin⁡θ≠0\sin\theta \neq 0. Case sin⁡θ=0\sin\theta = 0 (i.e. u=(±1,0)\mathbf u = (\pm 1, 0)). Then f(h,0)=h2⋅0h4=0f(h,0) = \dfrac{h^2 \cdot 0}{h^4} = 0 for all h≠0h \neq 0, so D(±1,0)f(0,0)=lim⁡h→00/h=0D_{(\pm1,0)}f(0,0) = \lim_{h\to0} 0/h = 0, consistent with the formula cos⁡2θ/sin⁡θ\cos^2\theta/\sin\theta having a removable-looking but genuinely undefined value at sin⁡θ=0\sin\theta=0 resolved directly to 00 by this direct computation. So every directional derivative exists at the origin. In particular the partials exist: at θ=0\theta = 0, D(1,0)f=0=∂f/∂x(0,0)D_{(1,0)}f = 0 = \partial f/\partial x(0,0); at θ=π/2\theta = \pi/2, D(0,1)f=cos⁡2(π/2)/1=0=∂f/∂y(0,0)D_{(0,1)}f = \cos^2(\pi/2)/1 = 0 = \partial f/\partial y(0,0). So if a gradient-dot-product formula held, we would need ∇f(0,0)=(0,0)\nabla f(0,0) = (0,0) and hence Duf(0,0)=0D_{\mathbf u}f(0,0) = 0 for every u\mathbf u. But at θ=π/4\theta = \pi/4: cos⁡2(π/4)=1/2\cos^2(\pi/4) = 1/2, sin⁡(π/4)=22\sin(\pi/4) = \tfrac{\sqrt2}{2}, so

Duf(0,0)=1/22/2=12≠0.D_{\mathbf u}f(0,0) = \frac{1/2}{\sqrt2/2} = \frac{1}{\sqrt2} \neq 0.

This contradicts what the (falsely assumed) gradient formula would require. The resolution: ff is not differentiable at the origin. (Indeed ff is not even continuous there: along the parabolic path y=x2y = x^2, f(x,x2)=x2⋅x2x4+x4=12f(x,x^2) = \dfrac{x^2\cdot x^2}{x^4+x^4} = \dfrac12 for all x≠0x \ne 0, while f(0,0)=0f(0,0)=0, so lim⁡(x,y)→(0,0)f(x,y)≠f(0,0)\lim_{(x,y)\to(0,0)} f(x,y) \ne f(0,0); differentiability requires continuity, which fails.) This shows the differentiability hypothesis in the Theorem cannot be weakened to "all directional derivatives exist": that weaker condition is compatible with a wildly non-linear, even discontinuous, directional-derivative profile.

Scope, edge cases & common traps

SituationWhat is trueWhat can go wrong
ff differentiable at p\mathbf pDuf(p)=∇f(p)⋅uD_{\mathbf u}f(\mathbf p) = \nabla f(\mathbf p)\cdot\mathbf u for all unit u\mathbf u; gradient gives steepest ascent—
ff has all partials, is C1C^1 near p\mathbf pff is automatically differentiable at p\mathbf p (standard sufficient condition), so the Theorem appliesStudents sometimes skip checking continuity of partials and wrongly assume differentiability from mere existence of partials
ff has all directional derivatives at p\mathbf p, not known differentiableNothing about linearity in u\mathbf u is guaranteedThe counterexample above: gradient-style reasoning simply fails
u\mathbf u not a unit vectorDufD_{\mathbf u}f as literally defined is not the same as ∇f⋅u\nabla f\cdot \mathbf u unless ∥u∥=1\|\mathbf u\|=1Forgetting to normalize u\mathbf u before applying the dot-product formula (see Example 1)
∇f(p)=0\nabla f(\mathbf p) = \mathbf 0All directional derivatives at p\mathbf p are zero; p\mathbf p is a critical pointConcluding p\mathbf p is automatically a local extremum — false in general (e.g. saddle points), addressed in Critical Points and Extrema
Level set orthogonalityHolds wherever ∇f(p)≠0\nabla f(\mathbf p) \ne \mathbf 0At points where ∇f(p)=0\nabla f(\mathbf p) = \mathbf 0, the level set may fail to be a smooth curve at all (e.g. self-intersections), and "orthogonal direction" is not even well posed

The mathematician behind it

Origins · 1 minSir William Rowan Hamilton and how this was discovered
SW
Sir William Rowan Hamilton
1805–1865 · Dublin, Ireland — mathematical physics and the calculus of vector operators

Hamilton, already renowned for his reformulation of mechanics (Hamiltonian dynamics), was among the pioneers who treated differential operators acting on scalar fields as objects in their own right. The symbol ∇\nabla (nabla), whose modern use as the gradient operator crystallized through the work of Hamilton and later Gibbs and Heaviside, encodes exactly the packaging of partial derivatives into a single vector-valued operation described in this article. Hamilton's broader program — treating operators algebraically, on par with numbers and vectors — set the stage for vector calculus as a coherent language, later systematized by Josiah Willard Gibbs and Oliver Heaviside for use in electromagnetism.

History
- 1837: Hamilton develops algebraic formulations of complex numbers and begins investigating operator methods in mechanics.
- 1843: Hamilton discovers the quaternions, providing an algebraic framework in which vector operators like ∇\nabla later find natural expression.
- 1846–1850s: Hamilton and collaborators use the nabla symbol for differential operators in early vector analysis, applied to physical fields.
- 1881–1884: Gibbs's Vector Analysis notes and Heaviside's electromagnetic papers isolate gradient, divergence, and curl as the modern trio of vector operators.
- Early 1900s: The gradient becomes standard vocabulary in calculus textbooks, formalized via the total derivative as presented here.
Why must u\mathbf u be a unit vector in the definition of the directional derivative?

The directional derivative is meant to measure a rate of change per unit distance traveled. If u\mathbf u were not normalized, the quantity lim⁡h→0f(p+hu)−f(p)h\lim_{h\to0}\frac{f(\mathbf p+h\mathbf u)-f(\mathbf p)}{h} would depend on the arbitrary length of u\mathbf u, scaling linearly with ∥u∥\|\mathbf u\| by linearity of the limit (replacing u\mathbf u with cuc\mathbf u multiplies the result by cc). Requiring ∥u∥=1\|\mathbf u\|=1 pins down a canonical, comparable notion of "speed of change while moving at unit speed," which is what makes the maximal-rate statement (∥∇f(p)∥\|\nabla f(\mathbf p)\| is the steepest rate) a meaningful, well-posed claim rather than something that could be inflated arbitrarily by rescaling the direction vector.

Does the gradient formula Duf=∇f⋅uD_{\mathbf u}f = \nabla f \cdot \mathbf u require ff to be C1C^1, or just differentiable at the single point p\mathbf p?

Only differentiability at the single point p\mathbf p is needed for the Theorem as proved here; the proof never uses behavior of ff at other points, nor continuity of the partials near p\mathbf p. However, differentiability at an isolated point without continuous partials nearby is a somewhat pathological, hard-to-construct situation. In practice, the much easier-to-check sufficient condition — all first partials exist and are continuous throughout an open neighborhood of p\mathbf p — is what analysts verify, since it implies differentiability at p\mathbf p (and at every point of that neighborhood) via a mean value theorem argument, which is a separate, standard result built directly on the definition of the total derivative.

Is the gradient basis-dependent? Does it change if I use different coordinates?

The directional derivative Duf(p)D_{\mathbf u}f(\mathbf p) is a geometric, coordinate-free quantity: it only depends on ff, p\mathbf p, and the direction u\mathbf u in space. The gradient vector ∇f(p)\nabla f(\mathbf p), as the vector of partial derivatives, is initially computed component-wise using standard Cartesian coordinates. If you change to a different orthonormal basis, the gradient's components transform exactly as a vector's components should (via the orthogonal change-of-basis matrix), so the vector itself — as a geometric arrow in space — does not change; only its coordinate representation does. In more general (non-orthonormal or curvilinear) coordinate systems, one must be more careful, and the gradient is more properly understood as the vector dual to the differential dfdf under the ambient inner product — a refinement studied in differential geometry.

What happens to the steepest-ascent theorem if ∇f(p)=0\nabla f(\mathbf p) = \mathbf 0?

If ∇f(p)=0\nabla f(\mathbf p) = \mathbf 0, the gradient formula still holds and gives Duf(p)=0⋅u=0D_{\mathbf u}f(\mathbf p) = \mathbf 0 \cdot \mathbf u = 0 for every unit vector u\mathbf u: the function is instantaneously flat in every direction from p\mathbf p. There is no well-defined "direction of steepest ascent" because every direction ties for the (trivial) maximum rate of 00. Such points are called critical points, and determining whether ff actually has a local max, local min, or saddle behavior there requires second-order information — the subject of Critical Points and Extrema, which builds on exactly this vanishing-gradient condition as its starting point.

Can the directional derivative exist without ff being continuous at that point?

Yes, and the counterexample in this article demonstrates precisely this: the function f(x,y)=x2y/(x4+y2)f(x,y) = x^2y/(x^4+y^2) (extended by 00 at the origin) has a directional derivative in every direction at the origin, yet fails to even be continuous there, since its limit along the parabola y=x2y=x^2 is 1/2≠f(0,0)=01/2 \ne f(0,0) = 0. This shows that the existence of directional derivatives is a comparatively weak hypothesis — much weaker than differentiability, and even weaker than continuity — which is exactly why the Theorem in this article insists on full differentiability rather than the mere existence of directional derivatives.

How does this relate to the total derivative and the chain rule?

The directional derivative is a special case of a more general fact: if γ(t)\gamma(t) is any differentiable curve with γ(0)=p\gamma(0)=\mathbf p, then ddtf(γ(t))∣t=0=∇f(p)⋅γ′(0)\frac{d}{dt}f(\gamma(t))\big|_{t=0} = \nabla f(\mathbf p)\cdot \gamma'(0), which is the multivariable chain rule. Taking γ(t)=p+tu\gamma(t) = \mathbf p + t\mathbf u for a unit vector u\mathbf u recovers exactly the directional derivative formula proved here. In fact, the proof technique used above (reduce to a one-variable limit, control the error via the differentiability remainder) is the same technique used to prove the chain rule itself, so this article's Theorem can be seen as the chain rule's most important special case, proved independently and without circularity.

Why is the gradient orthogonal to the level set rather than, say, tangent to it?

Along the level set f=cf = c, the function's value never changes, so the rate of change of ff along any tangent direction to that level set must be zero — that's the content of the orthogonality proof, which shows ∇f(p)⋅γ′(0)=0\nabla f(\mathbf p)\cdot \gamma'(0)=0 for any curve γ\gamma tracing the level set. Since ∇f(p)\nabla f(\mathbf p) is a fixed vector whose dot product with every tangent direction of the level set vanishes, and (in R2\mathbb R^2, generically) the level set's tangent line is one-dimensional, this pins the gradient down (up to scaling) to the unique direction perpendicular to that tangent line — hence orthogonal, not tangent, to the level set.

Before this

Where this leads