The Kramers-Moyal Expansion
Every continuous-time random process built from jumps — a molecule number changing by a reaction, a Brownian particle nudged by collisions, a stock price ticking — is governed exactly by a master equation, an integro-differential balance of probability flowing in and out of each state. That equation is exact but almost never solvable. The Kramers-Moyal expansion is the systematic bridge from that intractable exact law to the workhorse partial differential equations of stochastic physics: it Taylor-expands the master equation in the jump size and turns it into , an infinite-order PDE whose coefficients are the local moments of the jump distribution. The first two coefficients are the familiar ones: is the drift (the average velocity of the probability fluid) and is the diffusion (its spreading rate). Cutting the series after gives the Fokker-Planck equation — the engine behind Brownian motion, the Ornstein-Uhlenbeck process, and Kramers' escape-rate theory. But the expansion does something subtler than just produce Fokker-Planck: it tells you exactly what you threw away, and it warns you that you cannot keep just a little more. That warning is Pawula's theorem, the deep surprise of the subject. Because a probability density must stay non-negative, the even Kramers-Moyal coefficients are chained together by Cauchy-Schwarz inequalities. The consequence is stark: the series must either stop dead at or run forever. There is no consistent "improved" third- or fourth-order Fokker-Planck equation — a truncation at any finite order above two generically produces negative probabilities. Either your process is a genuine diffusion, or it carries an irreducible jump character that only the full master equation captures. This article derives the expansion from a single operator identity, proves Pawula's dichotomy, and demonstrates its teeth numerically.
- The Kramers-Moyal expansion rewrites the master equation as an infinite-order PDE , with coefficients that are rescaled jump moments.
- The first two coefficients are the drift and diffusion ; truncating at yields the Fokker-Planck equation.
- Pawula's theorem: a non-negative probability density forces the series to stop at or contain infinitely many terms — no finite truncation above second order is legitimate.
- The van Kampen system-size expansion makes the truncation quantitative: , so higher coefficients vanish relative to diffusion as the system grows.
- The expansion is the microscopic bridge unifying discrete jump processes (master equation), continuous diffusions (Langevin), and the Fokker-Planck operator.
- Key idea: expand the master equation's gain term in powers of the jump size — each order produces with the -th jump moment over ; drift and diffusion are , and everything else comes as an all-or-nothing package.
Expand the master equation's gain term in powers of the jump size: each order produces acting on , where is times the -th jump moment. Drift and diffusion are ; by Pawula's theorem everything above second order comes all-or-nothing.
Where this lives today
The Kramers-Moyal expansion is the working engine of data-driven stochastic modeling. Given an experimental time series — turbulent velocity increments, climate proxy records, heart-rate variability, financial returns — one estimates the conditional jump moments directly from short-time increments. If the fourth coefficient is found significantly nonzero relative to , Pawula's theorem certifies that a pure Fokker-Planck / Langevin model is inadequate and that jump processes must be included; if not, the reconstructed drift and diffusion define an effective Langevin equation for the system. This "Kramers-Moyal reconstruction" has become a standard nonparametric tool in complex-systems physics. In chemical and biological kinetics the expansion is the rigorous bridge from the exact chemical master equation to the deterministic rate equations (leading order in system size), the chemical Langevin equation (next order), and the linear-noise Fokker-Planck description of fluctuations. Whether intrinsic noise matters — whether a cell's gene-expression dynamics needs a stochastic treatment or a deterministic one suffices — is decided by exactly the scaling this article makes precise.
The physical picture
Picture the probability density as a fluid spread over the state axis. In a short interval each parcel takes a random step drawn from a local distribution. The mean of that step is a wind that advects the fluid — the drift . The variance smears it out — the diffusion . The skewness, kurtosis, and higher cumulants distort its shape in ever finer ways: those are . The Kramers-Moyal expansion is nothing more than the statement that the evolution operator is the average of the shift operator over the jump distribution, Taylor-expanded term by term. The shift operator moves a function back by — it is an exact identity, not an approximation. The master equation's gain term integrates , which is precisely acting on the product , averaged over the jump size . Expand the exponential, collect powers of , and each power becomes the moment — the coefficient . The infinite-order PDE is exact because the exponential series is exact. Why does the series usually stop at second order in practice? Because for a process built from small, frequent jumps, the moments scale as with the system size (van Kampen). Drift survives at order , diffusion at , and every higher coefficient is suppressed by an extra power of . In the large-system limit the Fokker-Planck truncation becomes asymptotically exact — the central-limit theorem made local. But there is a catch that no amount of care around the truncation can soften, and it is the whole point of the theory. Pawula's theorem says positivity of couples the even coefficients through Cauchy-Schwarz-type inequalities. You cannot keep a finite tail. Either the process is a genuine diffusion ( for ), or it has an irreducible jump character and the full series must be kept. The figure below shows what happens if you ignore this: a fourth-order "corrected Fokker-Planck" propagator dives below zero, while the exact master equation and the honest truncation both stay non-negative.
The core equations
The starting point is the master equation for a jump process on a continuous state . It is an exact balance: probability flows into from every other state at the rate , and flows out of at the rate . The difference is . This is exact for any Markov jump process, but it is an integral equation and rarely solvable in closed form.
where: — probability density of state ; — transition rate from to (probability per unit time); — final and initial states. Read it as a continuity statement: the density at changes by the imbalance of one-way currents on every link into and out of . The expansion converts this integral balance into a differential one. Writing the rate as a function of the starting point and the jump size , and Taylor-expanding the gain term about , produces an infinite-order PDE whose coefficients are the local jump moments.
where: — -th Kramers-Moyal coefficient (rescaled jump moment); — -th derivative with the alternating sign inherited from the shift operator . The term is absent because it cancels the loss term of the master equation exactly. Each coefficient is a conditional short-time jump moment: the average of the -th power of the displacement, per unit time, given the current state. This is the operational recipe used to extract the coefficients from data or from a microscopic rate.
where: — short time increment; — conditional average given the system is at ; the makes the coefficient of . In terms of the microscopic rate, — the rescaled -th moment of the jump distribution. Keeping only the first two terms gives the Fokker-Planck equation, the drift-diffusion PDE at the heart of continuous stochastic dynamics. This is the truncation that produces the Ornstein-Uhlenbeck process and Kramers' escape theory; the Fokker-Planck equation itself is developed in its own article as the law.
where: — drift coefficient (mean increment rate); — diffusion coefficient (half the mean-square increment rate). Note the placement: the drift sits inside one -derivative, the diffusion inside two. For thermal Brownian motion these are and , the Einstein relation. The legitimacy of stopping at is not a matter of taste — it is constrained by positivity. Because the transition density is a genuine probability, its moments obey Cauchy-Schwarz inequalities, which propagate to the coefficients as a chain of constraints. This is the Pawula constraint.
where: — positive integer indexing even orders; — a positive constant from the moment normalization. The consequence, proved below, is dramatic: if any even coefficient above vanishes, the entire tail vanishes — the series either stops at or runs to infinity. For a genuine thermal Langevin process the higher coefficients vanish identically, and the drift and diffusion take their Einstein-Smoluchowski values. This is the case in which the Fokker-Planck truncation is not an approximation but exact.
where: — mobility; — external force; — thermal energy; is the Einstein relation, recalled by name from the einstein-relation article. Gaussian white noise has vanishing moments beyond the second, so a Langevin process lives exactly at the Fokker-Planck level.
The goal is to prove that a probability-conserving jump process cannot be described by a Kramers-Moyal series truncated at any finite order between three and infinity. The strategy is to recognise that the short-time transition density is a genuine non-negative measure in the displacement , apply the Cauchy-Schwarz inequality to its moments, and translate the resulting inequality chain into a statement about the coefficients. The subtlety to watch: it is the even coefficients that anchor the chain, and a single vanishing even coefficient above the second forces every higher coefficient to zero.
Step 1. Treat the short-time transition density as a probability measure in the displacement ; it is non-negative, so its moments are moments of a genuine distribution. Why: positivity of is the only physical input, and it is exactly what the whole theorem hinges on.
Step 2. Apply Cauchy-Schwarz with weight by splitting : . Why: Cauchy-Schwarz is exact for any inner product, and the weight is precisely what makes a legitimate inner product.
Step 3. Translate to coefficients. Since , the same inequality holds up to constants: . Setting gives the even-order chain used in the core equations. Why: the moment-to-coefficient map is a fixed rescaling by , so the inequality survives intact.
Step 4. Watch the zero propagate. Suppose some even coefficient with . The inequality then forces the product with the left side zero, and applying the chain in both directions collapses the neighbours to zero as well. Iterating, every coefficient with must vanish. Why: a single interior zero in the even chain cannot coexist with nonzero neighbours without violating the inequality, so the tail is all-or-nothing.
Step 5. Conclude the dichotomy: either all (the process is a genuine diffusion described exactly by Fokker-Planck) or every for is nonzero (an irreducible jump process requiring the full series). There is no legitimate finite truncation at . Why: any such truncation sets a finite tail to zero while keeping a nonzero interior coefficient, which the inequality chain forbids for a non-negative density. *Verified: verify_aq-sm-kramers-moyal-expansion.py confirms the Cauchy-Schwarz chain numerically for a Gaussian weight, and demonstrates that the truncated propagator of a jump process develops negative values while the exact master equation and the Fokker-Planck propagator stay non-negative.*
Go deeper · Derivation B · 1 minFrom the master equation to Fokker-Planck via the shift operator
The goal is to derive the infinite-order Kramers-Moyal PDE, and its Fokker-Planck truncation, from the master equation using a single exact operator identity. The strategy rests on the shift operator: the gain term of the master equation is a shifted-and-averaged product, and the shift is implemented exactly by an exponential of the derivative. Expanding that exponential and integrating each power against the jump distribution generates the coefficients term by term; truncating at second order and identifying the moments delivers the Fokker-Planck equation with explicit drift and diffusion.
Step 1. Write the master equation with the jump variable : , where is the rate of a jump of size from . Why: changing variables from the final state to the jump exposes the gain term as a function shifted by , which is what the operator identity acts on.
Step 2. Implement the shift exactly. The operator satisfies , so . The term is , which cancels the loss term exactly. Why: the exponential-of-derivative identity is exact (it is the Taylor series), so no approximation has entered yet — the infinite-order PDE is equivalent to the master equation.
Step 3. Integrate each power of against the rate to form the moments: , leaving . Why: the -integral only touches the jump distribution, converting the dependence into the -th moment while the stays outside.
Step 4. Truncate at second order for a small-jump process, keeping and : . With and this is the Smoluchowski (overdamped Fokker-Planck) equation. Why: for small, frequent jumps the higher moments are suppressed by powers of the jump size — made quantitative by the van Kampen scaling — so the drift-diffusion pair dominates. If instead the jumps are symmetric, and only diffusion survives, recovering , the diffusion equation. *Verified: verify_aq-sm-kramers-moyal-expansion.py checks that a symmetric jump gives , , and , and that the Smoluchowski reduction yields , .*
Limiting cases
The behaviour of the Kramers-Moyal coefficients selects the type of process and the validity of the diffusion approximation:
| Case | Result | Meaning |
|---|---|---|
| Pure diffusion (Wiener / Langevin) | Only drift and diffusion survive; the expansion is exactly the Fokker-Planck equation with no error. Brownian motion and the Ornstein-Uhlenbeck process live here. | |
| Weak noise / large system size | As the higher coefficients vanish relative to diffusion; the Fokker-Planck truncation becomes asymptotically exact (van Kampen system-size expansion). | |
| Jump-dominated process | all comparable | Rare large jumps: the full master equation is needed; truncating at any finite order violates positivity (Pawula). Example: shot noise, chemical reactions away from . |
| Deterministic limit | When only the drift survives, giving the noiseless transport (Liouville) equation . |
Worked examples
Example 1 — The Kramers-Moyal coefficients of a symmetric jump process, and where Pawula bites. This computes the actual coefficients of a concrete process and locates the positivity failure.
Step 1. Take a process that jumps by or , each at rate . The -th jump moment is , which is for even and for odd .
Step 2. Form the coefficients . The drift (symmetric, no net wind). The diffusion . The first surviving higher coefficient is .
Step 3. For this gives and — a genuinely nonzero fourth coefficient, so by Pawula every higher coefficient is nonzero and no finite truncation is legitimate.
Step 4. Demonstrate the failure. Build the exact master-equation generator for lattice units (, ) and evolve a point source for a time . The exact propagator and the Fokker-Planck propagator both stay non-negative; the truncated propagator develops negative probability — the anti-diffusive term makes it unbounded and unphysical. The truncation at fourth order is not an approximation but a mathematical error.
Example 2 — Why the Fokker-Planck truncation is exact for thermal Brownian motion. This shows that a Langevin process sits precisely at .
Step 1. Take overdamped motion with unit-strength white noise .
Step 2. Compute the drift: , so .
Step 3. Compute the diffusion: , so by the Einstein relation.
Step 4. Compute the higher moments: for Gaussian white noise scales as for even and vanishes faster than , so identically. The Kramers-Moyal series terminates at second order exactly — the Fokker-Planck equation is not an approximation for thermal Brownian motion but the exact description, consistent with Pawula (the series stopped at two).
Origins · 1 minkramers-moyal expansion and how this was discovered
Going further
References. Risken, H. — The Fokker-Planck Equation: Methods of Solution and Applications, 2nd ed., Ch. 4 (Kramers-Moyal expansion, Pawula's theorem) · van Kampen, N. G. — Stochastic Processes in Physics and Chemistry, 3rd ed., Ch. VIII (system-size expansion) · Gardiner, C. W. — Handbook of Stochastic Methods, Ch. 7 · Pathria, R. K. & Beale, P. D. — Statistical Mechanics, 3rd ed., Ch. 15 · Kardar, M. — Statistical Physics of Particles, Ch. 9 · Reif, F. — Fundamentals of Statistical and Thermal Physics, Ch. 15 · Pawula, R. F. — Phys. Rev. 162, 186 (1967).
Provenance. Structure and coefficient definitions follow Risken (Ch. 4) and Gardiner (Ch. 7); system-size scaling per van Kampen (Ch. VIII); the Einstein relation and Brownian drift/diffusion per Pathria & Beale Ch. 15 and Reif Ch. 15. A verifier reproduces: (i) the shift-operator expansion giving ; (ii) Pawula's inequality via Cauchy-Schwarz and its cascade; (iii) the scaling; (iv) the Smoluchowski reduction , ; and (v) the numerical negative-probability demonstration for the truncation. No non-standard numerical values are asserted.
Builds on: The Master Equation · The Fokker-Planck Equation · Brownian Motion · Wiener Process and White Noise Leads to: The Fokker-Planck Equation · The Langevin Equation · Stochastic Calculus: Itô versus Stratonovich · Kramers' Escape Rate Theory · The Ornstein-Uhlenbeck Process
Is the Kramers-Moyal expansion an approximation or exact?
The full infinite series is exact — it is equivalent to the master equation, because the shift operator is an exact identity, not a perturbative expansion. What is approximate is the truncation. Cutting the series at (the Fokker-Planck equation) is an approximation for a jump process, exact for a genuine diffusion. The subtlety Pawula adds is that you may not truncate at any finite order between three and infinity: it is , or all of it.
What exactly is the difference between the master equation and the Kramers-Moyal expansion?
They describe the same dynamics. The master equation is an integral (gain-loss) balance over jumps; the Kramers-Moyal expansion is its differential form, obtained by Taylor-expanding the gain term in the jump size. The master equation is exact but usually intractable; the expansion is exact as an infinite series and becomes tractable when it can be truncated (at ). Think of it as trading an integral operator for a differential operator of infinite order.
Why does the sign alternate as ?
The alternating sign comes directly from the shift operator. Moving the argument backward by is , so each order carries . After integrating against the jump distribution to form , the structure remains. Physically, the odd terms (drift-like) advect and the even terms (diffusion-like) spread, and the signs keep the drift-diffusion structure of correct.
How do I actually extract the coefficients from experimental data?
Estimate the conditional short-time moments: for each value of the observable, collect all short-time increments that follow a visit to , and compute for small . Fit or extrapolate to . The drift and diffusion define an effective Langevin model; a significantly nonzero (relative to ) signals via Pawula that jumps are present and the Langevin picture is inadequate.
What is the van Kampen -expansion and how does it relate?
It is the rigorous version of "small jumps." For a system with an extensive variable (molecule number, spins) of size , one writes , separating a macroscopic trajectory from fluctuations. The jump moments then scale as , so for . This makes the Fokker-Planck truncation asymptotically exact as and produces the linear-noise (Ornstein-Uhlenbeck) description of fluctuations around the deterministic path.
If the propagator goes negative, does that mean the theory is broken?
No — it means the truncated equation is broken, which is exactly Pawula's point. The full master equation and the honest Fokker-Planck equation both preserve positivity. The operator is anti-diffusive (it amplifies high-wavenumber modes rather than damping them), so its propagator is not a probability. The lesson is prescriptive: never keep a lone fourth-order term. Either the process is diffusive and suffices, or it is genuinely a jump process and you must keep the full master equation.