The Multivariable Chain Rule

One-line statement: The multivariable chain rule computes the derivative of a composition of functions when intermediate quantities depend on several variables, summing contributions through every dependency path. Key result in words: The total rate of change equals the sum, over all intermediate variables, of (rate of change of the output w.r.t. that intermediate) × (rate of change of that intermediate w.r.t. the input). Where it applies: Functions

R^{n} \to R^{m}

composed with other multivariable functions; physics, optimization, machine learning (backpropagation), thermodynamics, differential geometry. Who/when: Rooted in Leibniz's and Newton's calculus (1670s); generalized to multiple variables through the 18th–19th century (Euler, Cauchy); formalized via the Jacobian by the late 1800s.

The multivariable chain rule extends the single-variable chain rule $\frac{d}{d x} f (g (x)) = f^{'} (g (x)) g^{'} (x)$ to functions where several variables depend on several other variables. In its most compact form, the derivative (Jacobian matrix) of a composition is the matrix product of the individual Jacobians. This matters because almost every quantity in applied mathematics is a composition: temperature depends on position, position depends on time, and we want temperature's rate of change in time. The single most surprising and powerful fact is that the entire structure — no matter how many intermediate layers — collapses to repeated matrix multiplication, which is precisely why backpropagation in deep neural networks works and is computationally efficient. Mastering it requires keeping ruthless track of which variable depends on which.

The Core Insight

A change in an input variable propagates to the output through every chain of dependencies. To find the total derivative, you trace each path from input to output, multiply the local derivatives along that path, and add up all the paths. The "sum over paths" is the heart of the matter; the matrix-product formulation is just this sum written compactly.

Formal Statement

Partial and Total Derivatives

Let $z = f (x_{1}, \dots, x_{n})$ be a differentiable function. The partial derivative $\frac{\partial f}{\partial x _{i}}$ measures the rate of change of $f$ as $x_{i}$ varies while all other $x_{j}$ ( $j \neq = i$ ) are held fixed.

If each $x_{i} = x_{i} (t)$ is a function of a single parameter $t$ , then $z$ becomes a function of $t$ alone, and its total derivative $\frac{d z}{d t}$ accounts for all the ways $t$ influences $z$ .

The Jacobian matrix of a map $f : R^{n} \to R^{m}$ , $f = (f_{1}, \dots, f_{m})$ , is the $m \times n$ matrix

D f = (\frac{\partial f _{i}}{\partial x _{j}})_{1 \leq i \leq m 1 \leq j \leq n} .

⟳stack the gradients

\nabla f_{i}

as rows; entry

(i, j)

is the sensitivity of output

f_{i}

to input

x_{j}

Multivariable Chain Rule — Scalar Output, Curve Input

Let $z = f (x_{1}, \dots, x_{n})$ be differentiable at a point, and let each $x_{i} = x_{i} (t)$ be differentiable. Then the composition $z (t) = f (x_{1} (t), \dots, x_{n} (t))$ is differentiable and

\frac{d z}{d t} = i = 1 \sum n \frac{\partial f}{\partial x _{i}} \frac{d x _{i}}{d t} = \nabla f \cdot \frac{d x}{d t} .

⟳take the total differential

d z = \sum_{i} \frac{\partial f}{\partial x _{i}} d x_{i}

and divide by

d t

; the dot product is gradient times velocity.

Multivariable Chain Rule — Several Independent Variables

Let $z = f (x_{1}, \dots, x_{n})$ where each $x_{i} = x_{i} (t_{1}, \dots, t_{k})$ . Then for each $j = 1, \dots, k$ ,

\frac{\partial z}{\partial t _{j}} = i = 1 \sum n \frac{\partial f}{\partial x _{i}} \frac{\partial x _{i}}{\partial t _{j}} .

⟳hold all

t_{ℓ}

(

ℓ \neq = j

) fixed and apply equation [2] with

t_{j}

playing the role of

t

General Form — Jacobian Composition

Let $g : R^{k} \to R^{n}$ be differentiable at $t_{0}$ and $f : R^{n} \to R^{m}$ be differentiable at $g (t_{0})$ . Then $h = f \circ g : R^{k} \to R^{m}$ is differentiable at $t_{0}$ and its Jacobian factors as a matrix product:

D h (t_{0}) = D f (g (t_{0})) \cdot D g (t_{0}) .

⟳the best linear approximation of a composition is the composition of best linear approximations; composing linear maps corresponds to multiplying their matrices.

The dimensions confirm consistency: $(m \times n) \cdot (n \times k) = (m \times k)$ , exactly the Jacobian of $h : R^{k} \to R^{m}$ .

Why It Holds / Derivation

Proof via Differentiability and Linear Approximation

We prove equation [2]; the general case [4] follows by applying it component-wise.

Step 1 — Differentiability hypothesis. Since

f

is differentiable at

a = x (t_{0})

, by definition there is a remainder function

ε (h)

with

ε (h) \to 0

h \to 0

such that

f (a + h) - f (a) = i = 1 \sum n \frac{\partial f}{\partial x _{i}} (a) h_{i} + ∥ h ∥ ε (h) .

⟳this is precisely the definition of (total) differentiability — linear part plus error vanishing faster than

∥ h ∥

Step 2 — Increment from $t$ . Let

h = x (t_{0} + Δ t) - x (t_{0})

, so

h_{i} = x_{i} (t_{0} + Δ t) - x_{i} (t_{0})

. Substituting into [5] and dividing by

Δ t

\frac{z ( t _{0} + Δ t ) - z ( t _{0} )}{Δ t} = i = 1 \sum n \frac{\partial f}{\partial x _{i}} (a) \frac{h _{i}}{Δ t} + \frac{∥ h ∥}{Δ t} ε (h) .

⟳divide both sides of [5] by the scalar

Δ t

and identify

z (t) = f (x (t))

Step 3 — Take the limit. As

Δ t \to 0

: each

\frac{h _{i}}{Δ t} \to x_{i}^{'} (t_{0})

by differentiability of

x_{i}

; the factor

\frac{∥ h ∥}{Δ t}

stays bounded (it tends to

∥ x^{'} (t_{0}) ∥

); and

ε (h) \to 0

since

h \to 0

. Hence the error term vanishes and

\frac{d z}{d t} (t_{0}) = i = 1 \sum n \frac{\partial f}{\partial x _{i}} (a) x_{i}^{'} (t_{0}) . ■

⟳product of (something bounded) and (something

\to 0

) is

\to 0

; the surviving sum is the claim.

Why differentiability — not just partial derivatives — is required

The proof crucially uses the differentiability of $f$ (Step 1), which is stronger than mere existence of partial derivatives. A function can have all partials at a point yet fail to be differentiable, and the chain rule can then break. Always verify differentiability (e.g., continuous partials suffice — class $C^{1}$ ).

Augustin-Louis Cauchy | 1789–1857

While Leibniz and Newton wielded chain-rule reasoning informally, Cauchy is central to the rigorous $ε$ – $δ$ foundation that makes the proof above legitimate. His insistence on precise definitions of limit and derivative transformed the chain rule from a symbolic manipulation of differentials into a theorem with hypotheses (differentiability) and a remainder analysis. The Jacobian matrix formulation that crowns the modern statement is named for Carl Gustav Jacob Jacobi (1804–1851), whose 1841 treatise De determinantibus functionalibus systematized functional determinants.

Building Intuition

Sum Over Dependency Paths

Draw a dependency graph: arrows from each input to each intermediate variable, then from each intermediate to the output. To find $\frac{\partial ( output )}{\partial ( input )}$ :

Find every directed path from the input node to the output node.
Along each path, multiply the local derivatives on its edges.
Sum over all paths.

If $z$ depends on $u, v$ , and both $u, v$ depend on $s, t$ , there are two paths from $s$ to $z$ (through $u$ , and through $v$ ), giving $\frac{\partial z}{\partial s} = \frac{\partial z}{\partial u} \frac{\partial u}{\partial s} + \frac{\partial z}{\partial v} \frac{\partial v}{\partial s}$ . The single-variable rule is the degenerate case of exactly one path.

Dependency graph for $z=f(u,v)$ with $u=u(s,t),\,v=v(s,t)$. Two paths carry $s$ to $z$.

Worked Examples

A particle's temperature along a path

Given: Temperature field

T (x, y) = x^{2} y + 3 y

(°C, with

x, y

in metres). A particle moves along

x (t) = cos t

y (t) = sin t

(metres), with

t

in seconds. Find

\frac{d T}{d t}

t = π /2

s. Step 1 — Partials.

\frac{\partial T}{\partial x} = 2 x y, \frac{\partial T}{\partial y} = x^{2} + 3.

⟳differentiate

x^{2} y + 3 y

treating the other variable as constant.

Step 2 — Velocities.

\frac{d x}{d t} = - sin t

\frac{d y}{d t} = cos t

. Step 3 — Chain rule [2].

\frac{d T}{d t} = (2 x y) (- sin t) + (x^{2} + 3) (cos t) .

⟳substitute the partials and velocities into

\frac{d T}{d t} = T_{x} x^{'} + T_{y} y^{'}

Step 4 — Evaluate at $t = π /2$ :

x = cos (π /2) = 0

y = sin (π /2) = 1

sin t = 1

cos t = 0

\frac{d T}{d t} = (2 \cdot 0 \cdot 1) (- 1) + (0^{2} + 3) (0) = 0.

⟳plug numbers; both terms vanish.

\frac{d T}{d t}_{t = π /2} = 0 °C/s

Polar-coordinate partial derivatives

Given:

f (x, y) = x^{2} + y^{2}

with

x = r cos θ

y = r sin θ

. Find

\frac{\partial f}{\partial r}

and

\frac{\partial f}{\partial θ}

r = 2

θ = π /4

. Step 1 — Partials of $f$ .

\frac{\partial f}{\partial x} = 2 x

\frac{\partial f}{\partial y} = 2 y

. Step 2 — Partials of $x, y$ .

\frac{\partial x}{\partial r} = cos θ, \frac{\partial y}{\partial r} = sin θ, \frac{\partial x}{\partial θ} = - r sin θ, \frac{\partial y}{\partial θ} = r cos θ .

⟳differentiate

r cos θ

and

r sin θ

w.r.t.

r

and

θ

Step 3 — Chain rule [3].

\frac{\partial f}{\partial r} = 2 x cos θ + 2 y sin θ = 2 r cos^{2} θ + 2 r sin^{2} θ = 2 r .

⟳substitute

x = r cos θ, y = r sin θ

and use

cos^{2} + sin^{2} = 1

\frac{\partial f}{\partial θ} = 2 x (- r sin θ) + 2 y (r cos θ) = - 2 r^{2} cos θ sin θ + 2 r^{2} sin θ cos θ = 0.

⟳the two terms cancel — consistent with

f = r^{2}

being independent of

θ

Step 4 — Evaluate at $r = 2$ :

\frac{\partial f}{\partial r} = 4

\frac{\partial f}{\partial θ} = 0

\frac{\partial f}{\partial r}_{r = 2} = 4, \frac{\partial f}{\partial θ} = 0

(Check:

f = r^{2} \Rightarrow \partial f / \partial r = 2 r = 4

. ✓)

Common Misconceptions

Distinguish $\frac{\partial z}{\partial t}$ from $\frac{dz}{dt}$

When $z = f (x, t)$ has an explicit $t$ and also $x = x (t)$ , the total derivative includes both the direct and indirect dependence:

\frac{d z}{d t} = \frac{\partial f}{\partial t} + \frac{\partial f}{\partial x} \frac{d x}{d t} .

⟳

t

reaches

z

directly (through the explicit slot) and indirectly (through

x

); sum both paths.

Forgetting the

\partial f / \partial t

term is the single most common error in physics and economics. The notation

\frac{D f}{D t}

(material derivative) in fluid dynamics is precisely this distinction made explicit.

Partials exist but the chain rule fails

Let

f (x, y) = ⎩ ⎨ ⎧ \frac{x y}{x ^{2} + y ^{2}} 0 (x, y) \neq = (0, 0) (x, y) = (0, 0) .

⟳a classic homogeneous-degree-zero example.

Both partials exist at the origin:

f_{x} (0, 0) = f_{y} (0, 0) = 0

(since

f \equiv 0

on both axes). Now take the path

x (t) = t

y (t) = t

. Then

f (t, t) = \frac{t ^{2}}{2 t ^{2}} = \frac{1}{2}

for

t \neq = 0

, so

z (t) = \frac{1}{2}

(constant), giving

\frac{d z}{d t} = 0

. The naive formula

f_{x} x^{'} + f_{y} y^{'} = 0 \cdot 1 + 0 \cdot 1 = 0

happens to agree here — but

f

is not even continuous at the origin (limit along

y = x

\frac{1}{2}

, along axes is

0

), so it is not differentiable, and the chain rule has no theorem to stand on. The agreement is a coincidence, and along other constructions the naive formula misfires. Moral: existence of partials does not license the chain rule.

Connections

Where the chain rule plugs into the wider theory

The multivariable chain rule is the computational engine behind several neighbouring results. The Gradient Vector appears as the row-vector factor $\nabla f$ in equation [2], and the rule shows $\frac{d z}{d t} = \nabla f \cdot v$ , directly yielding the Directional Derivative. The general Jacobian form [4] is the multivariable face of the Single-Variable Chain Rule, with matrix multiplication replacing scalar multiplication. It underlies the Implicit Function Theorem (differentiate a constraint $F (x, y) = 0$ along a curve), the change-of-variables formula in Multiple Integrals (through $det D g$ ), and the transformation of vector fields in Differential Geometry. In its iterated, layered form it is exactly the Backpropagation Algorithm.

Real-World Applications

Concrete uses across science and engineering

Backpropagation (deep learning): Gradients of a loss w.r.t. millions of weights are computed by repeated application of equation [4] backward through network layers — the reverse-mode autodiff that trains every modern neural network.
Thermodynamics: Relating partial derivatives of state functions under different held-fixed variables (e.g., converting between $C_{p}$ and $C_{v}$ ) uses chain-rule "triple product" identities.
Fluid dynamics: The material derivative $\frac{D u}{D t} = \frac{\partial u}{\partial t} + (u \cdot \nabla) u$ — the acceleration of a fluid parcel — is equation [14] in disguise, central to the Navier–Stokes equations.
Robotics & control: Mapping joint-space velocities to end-effector velocities uses the manipulator Jacobian, the $D g$ of the forward-kinematics map.
Economics: Comparative statics and the implicit differentiation of equilibrium conditions (e.g., how output responds to a price change through several channels) rely on multivariable chain-rule bookkeeping.
Computer graphics: Transforming surface normals and shading derivatives under coordinate or texture mappings uses Jacobian composition.
General relativity / coordinate changes: Tensor components transform via products of $\frac{\partial x ^{' μ}}{\partial x ^{ν}}$ — pure chain-rule Jacobians.

Historical Development

From Leibniz to backpropagation

1676 — Gottfried Wilhelm Leibniz develops differential notation $d y$ , $d x$ and manipulates differentials of composite quantities, implicitly using chain-rule reasoning.
1690s — Johann and Jakob Bernoulli apply chain-rule techniques to curves and the brachistochrone, treating differentials of functions of functions.
1755 — Leonhard Euler, in Institutiones calculi differentialis, systematizes partial differentiation and differentials of functions of several variables.
1797 — Joseph-Louis Lagrange introduces functional/prime notation, recasting the chain rule symbolically.
1821 — Augustin-Louis Cauchy's Cours d'analyse supplies rigorous limit definitions, enabling a proper proof with error terms.
1841 — Carl Gustav Jacob Jacobi publishes De determinantibus functionalibus, establishing the Jacobian determinant and matrix as the organizing object for transformations.
late 1800s — The matrix-product form of the chain rule [4] becomes standard as linear algebra and vector analysis mature (Gibbs, Heaviside).
1960s–1986 — Reverse-mode automatic differentiation is developed (Linnainmaa, 1970) and popularized as backpropagation (Rumelhart, Hinton, Williams, 1986), recognizing the chain rule as the foundation of neural-network training.

Subtleties & Limits

Edge cases and ongoing care

The clean theorem [4] requires differentiability at the relevant points; weakening to mere directional or partial differentiability invalidates it (see the counterexample). Subtle questions persist in: (a) non-smooth analysis — chain rules for subdifferentials and Clarke generalized gradients of nonsmooth functions require careful constraint qualifications, and a single clean equation does not always hold; (b) infinite dimensions — in Banach/Fréchet spaces the chain rule for Fréchet derivatives holds, but Gâteaux differentiability is too weak, mirroring the finite-dimensional cautions; (c) automatic differentiation correctness — efficiently and exactly differentiating programs with control flow, recursion, and nondifferentiable kinks (ReLU, $max$ ) is an active engineering and theoretical area, where the naive chain rule can yield wrong or undefined values at kinks. The conceptual rule is settled; its rigorous reach into nonsmooth and infinite-dimensional settings remains a refined subject.