Directional Derivatives and Differentiability

Directional derivatives measure the instantaneous rate of change of a multivariable function along a chosen direction; differentiability is the stronger condition that the function is well-approximated by a linear map at a point. Key result: differentiability implies all directional derivatives exist and equals the gradient dotted with the direction — but the converse is FALSE. Applies across multivariable calculus, optimization, physics, and machine learning. Crystallized in the 19th century through Cauchy, Weierstrass, and Fréchet.

The directional derivative of a function $f : R^{n} \to R$ at a point $a$ in the direction of a unit vector $u$ is the limit of difference quotients along the line through $a$ with direction $u$ . Differentiability at $a$ means there exists a single linear map (the derivative) such that $f$ is approximated to first order in every direction simultaneously — a fundamentally stronger requirement than the existence of directional derivatives one direction at a time. This distinction matters because nearly every theorem of multivariable calculus (chain rule, Taylor's theorem, the implicit function theorem) requires genuine differentiability, not mere existence of partials. The surprising fact that trips up most students: a function can possess every directional derivative at a point and still be discontinuous there. The gradient encapsulates differentiability when it holds, pointing in the direction of steepest ascent.

The Core Insight

Existence of directional derivatives is a one-dimensional condition checked along each line separately. Differentiability is a genuinely multidimensional condition: it demands that ONE linear map approximates $f$ uniformly as you approach $a$ from all directions at once. The gap between these — sequential 1D limits versus a single 2D/nD limit — is the source of nearly every subtlety on this page.

Formal Statement

Directional Derivative

Let $f : R^{n} \to R$ , let $a \in R^{n}$ , and let $u \in R^{n}$ be a unit vector ( $∥ u ∥ = 1$ ). The directional derivative of $f$ at $a$ in the direction $u$ is

D_{u} f (a) = h \to 0 lim \frac{f ( a + h u ) - f ( a )}{h}

⟳Restrict

f

to the line

g (h) = f (a + h u)

, then

D_{u} f (a) = g^{'} (0)

by the ordinary single-variable derivative definition.

Here $a = (a_{1}, \dots, a_{n})$ is the base point, $u = (u_{1}, \dots, u_{n})$ the direction, and $h \in R$ the scalar step. When $u = e_{i}$ (the $i$ -th standard basis vector), $D_{e_{i}} f (a)$ is the partial derivative $\partial f / \partial x_{i}$ .

Some authors do not require $u$ to be a unit vector. If $u$ is unnormalized, $D_{u} f$ scales linearly with $∥ u ∥$ , and the geometric interpretation as "rate per unit distance" is lost. Always check your textbook's convention. This page uses unit vectors.

Differentiability (Fréchet)

f : R^{n} \to R

is differentiable at

a

if there exists a linear map

L : R^{n} \to R

(equivalently a vector

\nabla f (a)

) such that

h \to 0 lim \frac{f ( a + h ) - f ( a ) - L ( h )}{∥ h ∥} = 0

⟳This says the error of the linear approximation

f (a) + L (h)

vanishes faster than

∥ h ∥

; it is the multivariable analogue of

f (a + h) = f (a) + f^{'} (a) h + o (h)

The linear map $L$ is the (total) derivative $D f (a)$ , and the vector representing it is the gradient $\nabla f (a) = (\frac{\partial f}{\partial x _{1}}, \dots, \frac{\partial f}{\partial x _{n}})$ , so that $L (h) = \nabla f (a) \cdot h$ .

Gradient Formula for Directional Derivatives

If $f$ is differentiable at $a$ , then every directional derivative exists and is given by

D_{u} f (a) = \nabla f (a) \cdot u = i = 1 \sum n \frac{\partial f}{\partial x _{i}} (a) u_{i}

⟳Set

h = h u

in equation [2], divide by

h

, and take

h \to 0

; the

o (∥ h ∥)

term vanishes because

∥ h ∥ = ∣ h ∣

Continuity from Differentiability

If $f$ is differentiable at $a$ , then $f$ is continuous at $a$ .

h \to 0 lim f (a + h) = f (a)

⟳From [2],

f (a + h) - f (a) = L (h) + o (∥ h ∥)

; both terms

\to 0

h \to 0

since

L

is linear hence continuous.

Direction of Steepest Ascent

Among all unit directions $u$ , the directional derivative $D_{u} f (a)$ is maximized when $u$ points along $\nabla f (a)$ , with maximum value $∥\nabla f (a) ∥$ .

∥ u ∥ = 1 max D_{u} f (a) = ∥\nabla f (a) ∥, attained at u = \frac{\nabla f ( a )}{∥\nabla f ( a ) ∥}

⟳By [3] and Cauchy–Schwarz,

\nabla f \cdot u = ∥\nabla f ∥ ∥ u ∥ cos θ = ∥\nabla f ∥ cos θ

, maximized at

θ = 0

Why It Holds / Derivation

Differentiability ⟹ Gradient Formula (Equation 3)

We prove that differentiability at $a$ forces $D_{u} f (a) = \nabla f (a) \cdot u$ for every unit $u$ .

Step 1. By definition [2], write the remainder as

r (h) = f (a + h) - f (a) - \nabla f (a) \cdot h,

with the property

r (h) /∥ h ∥ \to 0

h \to 0

. Step 2. Specialize to

h = h u

with

h \in R

h \neq = 0

. Since

∥ u ∥ = 1

, we have

∥ h ∥ = ∣ h ∣

. Then

f (a + h u) - f (a) = \nabla f (a) \cdot (h u) + r (h u) = h (\nabla f (a) \cdot u) + r (h u) .

Step 3. Divide by

h

\frac{f ( a + h u ) - f ( a )}{h} = \nabla f (a) \cdot u + \frac{r ( h u )}{h} .

Step 4. Estimate the remainder term. Since

∣ h ∣ = ∥ h u ∥

\frac{r ( h u )}{h} = \frac{∣ r ( h u ) ∣}{∥ h u ∥} h \to 0 0

by the defining property of

r

in Step 1. Step 5. Taking

h \to 0

in Step 3 yields the limit

D_{u} f (a) = \nabla f (a) \cdot u

, which is equation [3].

■

Sufficient Condition for Differentiability

If all partial derivatives $\partial f / \partial x_{i}$ exist in a neighborhood of $a$ and are continuous at $a$ , then $f$ is differentiable at $a$ . (Such $f$ is called continuously differentiable, or $C^{1}$ .)

Sketch: Continuous Partials ⟹ Differentiable (n=2)

Step 1. Write the increment as a telescoping sum, changing one coordinate at a time:

f (a_{1} + h_{1}, a_{2} + h_{2}) - f (a_{1}, a_{2}) = [f (a_{1} + h_{1}, a_{2} + h_{2}) - f (a_{1}, a_{2} + h_{2})] + [f (a_{1}, a_{2} + h_{2}) - f (a_{1}, a_{2})] .

Step 2. Apply the single-variable Mean Value Theorem to each bracket. There exist

ξ_{1}

between

a_{1}

and

a_{1} + h_{1}

, and

ξ_{2}

between

a_{2}

and

a_{2} + h_{2}

, with

= \frac{\partial f}{\partial x _{1}} (ξ_{1}, a_{2} + h_{2}) h_{1} + \frac{\partial f}{\partial x _{2}} (a_{1}, ξ_{2}) h_{2} .

Step 3. Use continuity of the partials: as

h \to 0

, the evaluation points approach

a

, so

\frac{\partial f}{\partial x _{i}} (point) = \frac{\partial f}{\partial x _{i}} (a) + ε_{i} (h), ε_{i} (h) \to 0.

Step 4. Substitute: the increment equals

\nabla f (a) \cdot h + ε_{1} h_{1} + ε_{2} h_{2}

. The error term satisfies

\frac{∣ ε _{1} h _{1} + ε _{2} h _{2} ∣}{∥ h ∥} \leq ∣ ε_{1} ∣ + ∣ ε_{2} ∣ \to 0,

verifying [2].

■

Maurice Fréchet | 1878–1973

The French mathematician Maurice Fréchet gave the modern, rigorous definition of differentiability in normed and metric spaces (1911), generalizing the classical multivariable derivative to infinite-dimensional settings. The "Fréchet derivative" of equation [2] is now the standard definition. Fréchet also pioneered abstract metric spaces and functional analysis, providing the language in which "differentiability as approximation by a bounded linear operator" became precise. His framework distinguishes the strong (Fréchet) derivative from the weaker Gâteaux derivative, which is exactly the directional-derivative notion lifted to function spaces — making this very page's central tension a foundational theme of modern analysis.

Building Intuition

Imagine standing on a hillside whose height is $f (x, y)$ . A directional derivative is the slope you feel if you start walking in a fixed compass direction $u$ — steepness measured along that one path. There are infinitely many such slopes, one per direction.

Differentiability is a much stronger claim: it says the hillside near your feet looks like a flat tilted plane (the tangent plane). If that's true, then every walking-direction's slope is just the projection of one master arrow — the gradient — onto your direction. The gradient points "straight uphill," and its length is the steepest possible slope.

The crucial subtlety: a surface can have a well-defined slope along every straight-line path through a point and yet still be crinkled, creased, or torn there — so no single tangent plane fits. Directional derivatives only probe along straight lines; differentiability demands good behavior along every approach, including curved and oscillating ones. That's why all directional derivatives existing is not enough.

Gradient and directional derivatives on a level-set map. The gradient is perpendicular to level curves and gives the steepest slope; a directional derivative projects it onto a chosen direction.

<svg viewBox="0 0 460 360" xmlns="http://www.w3.org/2000/svg">
<rect width="460" height="360" fill="#0a0e17"/>
<!-- level curves -->
<ellipse cx="200" cy="180" rx="40" ry="28" fill="none" stroke="#94a3b8" stroke-width="1"/>
<ellipse cx="200" cy="180" rx="80" ry="56" fill="none" stroke="#94a3b8" stroke-width="1"/>
<ellipse cx="200" cy="180" rx="120" ry="84" fill="none" stroke="#94a3b8" stroke-width="1"/>
<ellipse cx="200" cy="180" rx="160" ry="112" fill="none" stroke="#94a3b8" stroke-width="1"/>
<text x="20" y="30" fill="#e2e8f0" font-size="13" font-family="sans-serif">Level curves f = const</text>
<!-- base point -->
<circle cx="200" cy="124" r="4" fill="#d4a853"/>
<text x="208" y="120" fill="#e2e8f0" font-size="12" font-family="sans-serif">a</text>
<!-- gradient arrow (steepest ascent, perpendicular to level curve, pointing inward/up) -->
<line x1="200" y1="124" x2="200" y2="64" stroke="#d4a853" stroke-width="2.5"/>
<polygon points="200,56 195,68 205,68" fill="#d4a853"/>
<text x="208" y="72" fill="#d4a853" font-size="12" font-family="sans-serif">∇f(a)  (steepest)</text>
<!-- direction u -->
<line x1="200" y1="124" x2="262" y2="100" stroke="#e2e8f0" stroke-width="2"/>
<polygon points="268,98 256,98 261,108" fill="#e2e8f0"/>
<text x="270" y="98" fill="#e2e8f0" font-size="12" font-family="sans-serif">u</text>
<!-- angle theta -->
<path d="M 200 104 A 20 20 0 0 1 218 113" fill="none" stroke="#94a3b8" stroke-width="1"/>
<text x="214" y="100" fill="#94a3b8" font-size="11" font-family="sans-serif">θ</text>
<!-- formula -->
<text x="20" y="335" fill="#e2e8f0" font-size="13" font-family="sans-serif">D_u f(a) = ∇f(a)·u = |∇f| cos θ</text>
</svg>

Worked Examples

Example 1 — Computing a Directional Derivative

Given:

f (x, y) = x^{2} y + 3 y^{2}

at the point

a = (1, 2)

, in the direction of

v = (3, 4)

. Step 1 — Normalize the direction.

∥ v ∥ = 3^{2} + 4^{2} = 5

, so

u = (\frac{3}{5}, \frac{4}{5})

. Step 2 — Compute the gradient.

\frac{\partial f}{\partial x} = 2 x y, \frac{\partial f}{\partial y} = x^{2} + 6 y .

(1, 2)

\partial f / \partial x = 2 (1) (2) = 4

and

\partial f / \partial y = 1^{2} + 6 (2) = 13

. So

\nabla f (1, 2) = (4, 13)

. Step 3 — Apply equation [3].

D_{u} f (1, 2) = \nabla f \cdot u = 4 \cdot \frac{3}{5} + 13 \cdot \frac{4}{5} = \frac{12}{5} + \frac{52}{5} = \frac{64}{5} .

(Since $f$ is a polynomial, it is $C^{1}$ everywhere, so equation [3] is valid.)

Answer:

D_{u} f (1, 2) = \frac{64}{5} = 12.8 (units of f per unit length)

Example 2 — Steepest Ascent and a Temperature Field

Given: A metal plate has temperature

T (x, y) = 100 - x^{2} - 2 y^{2}

(°C), with

x, y

in centimeters. Find the direction of fastest temperature increase at

(2, 1)

and the maximum rate. Step 1 — Gradient.

\frac{\partial T}{\partial x} = - 2 x, \frac{\partial T}{\partial y} = - 4 y \Rightarrow \nabla T (2, 1) = (- 4, - 4) .

Step 2 — Direction of steepest ascent (equation [5]) is along

\nabla T

u = \frac{( - 4 , - 4 )}{∥ ( - 4 , - 4 ) ∥} = \frac{( - 4 , - 4 )}{4 2} = (- \frac{1}{2}, - \frac{1}{2}) .

Step 3 — Maximum rate is

∥\nabla T ∥

∥\nabla T (2, 1) ∥ = (- 4)^{2} + (- 4)^{2} = 32 = 42 \approx 5.66.

Answer:

Steepest ascent toward (- \frac{1}{2}, - \frac{1}{2}), rate = 42 \approx 5.66 \frac{°C}{cm}

Heat flows opposite to this, along

(\frac{1}{2}, \frac{1}{2})

(down the gradient).

Common Misconceptions

"If all partial derivatives exist, the function is differentiable." FALSE. Partials only probe the coordinate-axis directions. "If all directional derivatives exist (in every direction), the function is differentiable." ALSO FALSE — and even worse, such a function need not be continuous. Differentiability requires the single 2D limit [2] to vanish, which is strictly stronger than the existence of any collection of 1D limits. The correct sufficient condition is continuous partial derivatives (

C^{1}

), not mere existence.

Directional Derivatives Exist Everywhere, Yet Not Differentiable / Not Continuous

Define

f (x, y) = ⎩ ⎨ ⎧ \frac{x ^{2} y}{x ^{4} + y ^{2}}, 0, (x, y) \neq = (0, 0), (x, y) = (0, 0) .

Directional derivatives all exist at the origin. For a unit direction

u = (a, b)

with

b \neq = 0

D_{u} f (0, 0) = h \to 0 lim \frac{1}{h} \cdot \frac{( ha ) ^{2} ( hb )}{( ha ) ^{4} + ( hb ) ^{2}} = h \to 0 lim \frac{a ^{2} b h ^{3}}{h ^{2} ( h ^{2} a ^{4} + b ^{2} )} = \frac{a ^{2} b}{b ^{2}} = \frac{a ^{2}}{b} .

For

b = 0

(the

x

-axis),

f (h, 0) = 0

, so

D_{u} f (0, 0) = 0

. Every directional derivative exists. But $f$ is not even continuous at the origin. Approach along the parabola

y = x^{2}

f (x, x^{2}) = \frac{x ^{2} \cdot x ^{2}}{x ^{4} + x ^{4}} = \frac{x ^{4}}{2 x ^{4}} = \frac{1}{2} \neq = 0 = f (0, 0) .

The limit depends on the path, so

f

is discontinuous — hence (by equation [4]) not differentiable, despite all directional derivatives existing. Moral: the directional derivatives, even if they existed and satisfied

D_{u} f = \nabla f \cdot u

, would not guarantee differentiability. Straight-line probing misses the parabolic ridge.

Partials Exist but Function is Discontinuous

A simpler classic:

g (x, y) = ⎩ ⎨ ⎧ \frac{x y}{x ^{2} + y ^{2}}, 0, (x, y) \neq = (0, 0), (x, y) = (0, 0) .

Both partials exist at the origin (

g (x, 0) = g (0, y) = 0

, so

g_{x} (0, 0) = g_{y} (0, 0) = 0

), yet

g (t, t) = \frac{1}{2} \to \frac{1}{2} \neq = 0

along

y = x

. So

g

is discontinuous, hence not differentiable, even though both partials exist. Existence of partials says nothing about differentiability.

Connections

Directional derivatives and differentiability sit at the hub of multivariable analysis. The gradient they define is the workhorse of Gradient Descent in optimization and machine learning. Equation [3] is the seed of the Multivariable Chain Rule, which expresses $\frac{d}{d t} f (r (t)) = \nabla f \cdot r^{'} (t)$ — a directional derivative along a curve. The condition [2] generalizes directly to the Total Derivative (Jacobian) for vector-valued maps $R^{n} \to R^{m}$ , and onward to the Fréchet Derivative in Banach spaces, where the directional version becomes the Gâteaux Derivative. Geometrically, the vanishing of all directional derivatives at an interior extremum gives the first-order condition $\nabla f = 0$ , linking to Lagrange Multipliers and critical-point theory. The orthogonality of $\nabla f$ to level sets connects to Tangent Planes and Linear Approximation and the Implicit Function Theorem.

Real-World Applications

Machine learning / neural networks: Backpropagation computes the gradient of a loss function over millions of parameters; gradient descent steps in the direction $- \nabla L$ (steepest descent, equation [5]) to minimize loss.
Image processing: The gradient magnitude $∥\nabla I ∥$ of an intensity field detects edges (Sobel and Canny edge detectors); the gradient direction gives edge orientation.
Geophysics & cartography: Slope and aspect of terrain are literally the magnitude and direction of $\nabla (elevation)$ ; used in hydrology to model water flow (steepest descent).
Thermodynamics & heat transfer: Fourier's law $q = - k \nabla T$ states heat flux flows down the temperature gradient — Example 2 made concrete.
Fluid dynamics & electrostatics: Conservative force fields and electric fields are gradients of potentials, $E = - \nabla ϕ$ ; directional derivatives give work done along paths.
Robotics / path planning: Artificial potential field methods steer robots along $- \nabla U$ toward goals and away from obstacles.
Economics: Marginal rates of substitution and the direction of fastest utility increase are directional derivatives of utility functions along consumption-bundle directions.

Historical Development

1797 — Lagrange: Théorie des fonctions analytiques develops partial derivatives systematically and introduces the notation $f^{'}$ , treating derivatives as coefficients of Taylor expansions.
1821 — Cauchy: Cours d'analyse puts limits and continuity on rigorous footing, giving the modern definition of the derivative via difference quotients — the basis of equation [1].
1840s–1870s — Weierstrass: The $ε$ – $δ$ formulation of limits makes precise what "approximation to first order" means, enabling the rigorous remainder condition [2]; his pathological functions warn against naive intuition.
1873 — Hamilton & the nabla: The operator $\nabla$ , introduced by Hamilton and popularized by Tait, gives the gradient its modern symbol and vector-calculus identity.
1881 — Gibbs & Heaviside: Develop vector analysis as we know it, formalizing $\nabla f$ , dot products, and the directional-derivative formula [3] for physics applications.
1887 — Peano: Provides explicit counterexamples showing existence of partials does not imply differentiability or continuity, sharpening the conceptual distinction.
1911–1925 — Fréchet: Defines the (strong) derivative for general normed spaces via condition [2], establishing the modern abstract notion of differentiability.
1913 — Gâteaux: Introduces the weaker directional-derivative-style derivative in function spaces, posthumously named the Gâteaux derivative, formalizing the directional vs. total distinction at the highest level.
20th century — optimization era: Cauchy's steepest-descent idea (1847) blossoms into modern numerical optimization and, ultimately, deep-learning training.

Subtleties & Limits

Several deep issues hover around this seemingly elementary topic. (1) The gap between Gâteaux and Fréchet differentiability is enormous in infinite dimensions: a functional can be Gâteaux-differentiable everywhere yet Fréchet-differentiable nowhere, and characterizing where Lipschitz functions on Banach spaces are differentiable is the subject of deep results (Preiss's theorem, 1990) and still-open questions in the geometry of Banach spaces. (2) Differentiability of Lipschitz functions: Rademacher's theorem guarantees a Lipschitz $f : R^{n} \to R$ is differentiable almost everywhere, but the structure of the (measure-zero) non-differentiability set is subtle — related to open problems on "universal differentiability sets." (3) Weak derivatives and Sobolev spaces replace pointwise directional derivatives with distributional ones, the foundation of modern PDE theory, where "differentiability" is reinterpreted entirely. (4) Even in finite dimensions, finding clean necessary-and-sufficient pointwise conditions for differentiability (beyond the sufficient $C^{1}$ criterion) remains genuinely delicate; existence of all directional derivatives plus continuity still does not suffice in general.

Directional Derivatives and Differentiability

The Core Insight

Formal Statement

Directional Derivative

Differentiability (Fréchet)

Gradient Formula for Directional Derivatives

Continuity from Differentiability

Direction of Steepest Ascent

Why It Holds / Derivation

Differentiability ⟹ Gradient Formula (Equation 3)

Sufficient Condition for Differentiability

Sketch: Continuous Partials ⟹ Differentiable (n=2)

Maurice Fréchet | 1878–1973

Building Intuition

Gradient and directional derivatives on a level-set map. The gradient is perpendicular to level curves and gives the steepest slope; a directional derivative projects it onto a chosen direction.

Worked Examples

Example 1 — Computing a Directional Derivative

Example 2 — Steepest Ascent and a Temperature Field

Common Misconceptions

Directional Derivatives Exist Everywhere, Yet Not Differentiable / Not Continuous

Partials Exist but Function is Discontinuous

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also

Before this

Where this leads