Partial Derivatives and the Gradient

A partial derivative of a function $f(x_1,\dots ,x_n)$ with respect to $x_i$ is the ordinary derivative obtained by treating all variables except $x_i$ as constants. The gradient $\nabla f$ collects all $n$ first-order partial derivatives into a single vector field, encoding the complete first-order local behavior of $f$. These tools matter because they generalize the single-variable derivative to functions of many variables, which describe nearly every physical, economic, and computational system of interest. A surprising fact: although each partial derivative looks at only one coordinate axis, the gradient — built entirely from these axis-aligned slopes — nonetheless identifies the globally optimal direction of increase among all possible directions, a fact that is not obvious and is the engine behind gradient-based optimization.

The Core Insight

Differentiation in many dimensions splits into two complementary objects. Partial derivatives are scalar slopes along coordinate axes. The gradient is the vector that organizes them so that, for any unit direction $\mathbf{u}$, the rate of change of $f$ in that direction is simply $\nabla f\cdot \mathbf{u}$. The single dot product collapses an infinite family of directional questions into one linear algebra operation — and immediately reveals that the steepest ascent direction is $\nabla f$ itself.

Formal Statement

Partial Derivative

Let $f:{\mathbb{R}}^n\to \mathbb{R}$ and let $\mathbf{a}=(a_1,\dots ,a_n)$. The partial derivative of $f$ with respect to $x_i$ at $\mathbf{a}$ is

The Gradient

For a differentiable $f:{\mathbb{R}}^n\to \mathbb{R}$, the gradient is the vector of all first partials:

Directional Derivative

The rate of change of $f$ at $\mathbf{a}$ in the direction of a unit vector $\mathbf{u}$ is

Steepest Ascent and Orthogonality to Level Sets

Let $f$ be differentiable at $\mathbf{a}$ with $\nabla f(\mathbf{a})\ne 0$. Then:

1. The maximum value of $D_{\mathbf{u}}f(\mathbf{a})$ over all unit $\mathbf{u}$ equals $\Vert \nabla f(\mathbf{a})\Vert$, attained when $\mathbf{u}=\nabla f(\mathbf{a})/\Vert \nabla f(\mathbf{a})\Vert$.

1. $\nabla f(\mathbf{a})$ is orthogonal to the level set $\{f=f(\mathbf{a})\}$ at $\mathbf{a}$.

Why It Holds / Derivation

Gradient gives directional derivative and steepest ascent

where $\theta$ is the angle between $\nabla f(\mathbf{a})$ and $\mathbf{u}$. Since $\cos \theta \in [-1,1]$, the maximum $\Vert \nabla f(\mathbf{a})\Vert$ occurs at $\theta =0$ (steepest ascent) and the minimum $-\Vert \nabla f(\mathbf{a})\Vert$ at $\theta =\pi$ (steepest descent). This proves [4]. The orthogonality claim [5] follows because tangent directions to the level set correspond to $\theta =\pi /2$ where the directional derivative vanishes. $■$

Differentiability is required, not just existence of partials

Equations [3]–[5] assume $f$ is differentiable at $\mathbf{a}$, meaning $f$ admits a good linear approximation. The mere existence of all partial derivatives does not guarantee differentiability, and in such pathological cases the formula $D_{\mathbf{u}}f=\nabla f\cdot \mathbf{u}$ can fail. A standard sufficient condition: if all partials exist and are continuous in a neighborhood of $\mathbf{a}$ (i.e. $f\in C^1$), then $f$ is differentiable there.

Carl Gustav Jacob Jacobi | 1804–1851

Jacobi systematized partial differentiation through his work on the theory of multiple-variable transformations, introducing the Jacobian determinant that bears his name and embedding partial derivatives into a rigorous matrix framework. His treatment of functional dependence and changes of variable made the array of partial derivatives — what we now call the Jacobian matrix, whose single-output special case is the gradient — a first-class mathematical object. Jacobi's emphasis on the structured organization of derivatives directly anticipated the modern vector gradient and the broader machinery of differential calculus on ${\mathbb{R}}^n$.

Building Intuition

Picture $f(x,y)$ as the elevation of a mountainous terrain over a map. A partial derivative $\partial f/\partial x$ is the slope you feel if you walk due east, ignoring north–south tilt — it is one specific compass-aligned slope. But terrain doesn't care about compass directions; the steepest uphill might point northeast. The gradient is the arrow on your map that points directly uphill in the steepest possible direction, and its length tells you how steep that climb is. Crucially, the gradient is built only from the east–west and north–south slopes, yet it correctly identifies the best diagonal direction — because any direction's slope is a weighted blend of the axis slopes, and that blend is exactly a dot product. Water flowing downhill always runs anti-parallel to the gradient, and contour lines on a topographic map (level sets) always cross the gradient at right angles.

Gradient vector perpendicular to level curves, pointing toward steepest ascent

Worked Examples

Gradient of a paraboloid and steepest ascent

Temperature gradient — heat flux direction

Common Misconceptions

Frequent errors with partials and gradients

• The gradient is not a slope (scalar); it is a vector. Its magnitude is the maximal slope.
• Do not confuse $\nabla f$ (gradient, input on a scalar field) with the divergence $\nabla \cdot \mathbf{F}$ or curl $\nabla \times \mathbf{F}$, which act on vector fields and produce different objects.
• Mixed partials need not be equal in pathological cases. Clairaut's theorem guarantees ${\partial }_{xy}f={\partial }_{yx}f$ only when those mixed partials are continuous.
• Existence of partials ≠ continuity ≠ differentiability. All four properties (partials exist, continuity, differentiability, $C^1$) are genuinely distinct.
• The dot-product formula $D_{\mathbf{u}}f=\nabla f\cdot \mathbf{u}$ requires $\mathbf{u}$ to be a unit vector; using a non-unit vector silently rescales the rate.

Partials exist but $f$ is not even continuous

Consider

Both partials exist at the origin: $f_x(0,0)={\lim }_{h\to 0}\frac{f(h,0)-0}{h}=0$ and likewise $f_y(0,0)=0$. Yet $f$ is not continuous at $(0,0)$ — approaching along $y=x$ gives $f\to \frac{1}{2}$, but along the axes $f=0$. Hence the partials existing tells us nothing about continuity, let alone differentiability. The naive formula $D_{\mathbf{u}}f=\nabla f\cdot \mathbf{u}=0$ would falsely predict zero change in every direction.

Connections

The gradient is the transpose / vector form of the Jacobian Matrix for the special case of a scalar-valued function. It is the central ingredient of the Multivariable Chain Rule, which expresses $\frac{d}{dt}f(\mathbf{r}(t))=\nabla f\cdot {\mathbf{r}}^{\prime }(t)$. It underlies the Method of Lagrange Multipliers, where constrained optima require $\nabla f=\lambda \nabla g$ (gradients parallel). In vector calculus, $\nabla$ generalizes to the Divergence and Curl operators, and the gradient is the first member of the differential-operator family unified by the Exterior Derivative. The condition $\nabla f=0$ defines Critical Points and the Second Derivative Test.

Real-World Applications

1. Machine learning — gradient descent. Neural networks minimize a loss $L(\mathbf{w})$ by repeatedly stepping $\mathbf{w}\leftarrow \mathbf{w}-\eta \nabla L$, the single most-used optimization rule in modern AI.
2. Thermodynamics & heat transfer. Fourier's law $\mathbf{q}=-k\nabla T$ states heat flux is proportional to the negative temperature gradient.
3. Electromagnetism. The electric field is the negative gradient of potential, $\mathbf{E}=-\nabla V$, converting a scalar potential map into a force field.
4. Fluid dynamics. Pressure gradients $\nabla p$ drive flow; the term appears directly in the Navier–Stokes momentum equation.
5. Computer graphics & vision. Image gradients detect edges (Sobel/Canny operators) by locating regions of large $\Vert \nabla I\Vert$.
6. Economics. The gradient of a utility or production function gives marginal rates and underlies marginal analysis and constrained optimization.
7. Geophysics & GIS. Terrain slope and aspect, watershed delineation, and steepest-descent flow routing all compute elevation gradients on digital elevation models.

Historical Development

• 1740s — Euler develops systematic partial differentiation in the study of functions of several variables and the calculus of variations.
• 1740 — Clairaut establishes the equality of mixed partial derivatives under continuity (now Clairaut's / Schwarz's theorem), in work on the shape of the Earth.
• 1770s — Lagrange introduces multiplier methods for constrained optimization, implicitly using gradient parallelism.
• 1797 — Lagrange popularizes prime/subscript derivative notation in Théorie des fonctions analytiques.
• 1841 — Jacobi publishes his theory of functional determinants, formalizing arrays of partial derivatives (the Jacobian).
• 1843 — Hamilton introduces the operator $\nabla$ ("nabla") in connection with quaternions.
• 1873 — Maxwell employs and refines $\nabla$ notation in his Treatise on Electricity and Magnetism, including the gradient.
• 1880s — Gibbs and Heaviside develop modern vector calculus, cementing $\nabla f$, divergence, and curl as standard notation taught today.
• 1847 — Cauchy describes the method of steepest descent, the conceptual ancestor of gradient-based optimization.

Subtleties & Limits

Several conceptual subtleties extend beyond the introductory picture. (1) Coordinate dependence: the naive component definition of $\nabla f$ is correct only in orthonormal Cartesian coordinates; in curvilinear or non-Euclidean settings the gradient requires the metric tensor, $(\nabla f{)}^i=g^{ij}{\partial }_{j}f$, distinguishing the gradient vector from the differential $df$ (a covector). This distinction — vectors versus covectors — is invisible in ${\mathbb{R}}^n$ with the standard metric but essential in differential geometry and general relativity. (2) Non-smoothness: for non-differentiable convex functions (e.g. $\Vert \mathbf{x}{\Vert }_1$ in machine learning) the gradient is replaced by the subgradient, and optimization theory must handle set-valued generalizations. (3) Infinite dimensions: generalizing the gradient to function spaces leads to the Fréchet and Gâteaux derivatives, with delicate existence questions central to the calculus of variations and PDE theory.

See Also