Stochastic Processes

Stochastic processes are collections of random variables indexed by time (or space), modeling systems that evolve under uncertainty. Key result: a process is fully characterized by its finite-dimensional distributions (Kolmogorov), and important classes — Markov chains, Poisson processes, Brownian motion — admit tractable analysis. Where it applies: finance, queueing, physics, biology, signal processing. Who/when: foundations laid by Markov (1906), Wiener (1923), and Kolmogorov (1933).

A stochastic process is a family ${X_{t}}_{t \in T}$ of random variables defined on a common probability space, indexed by a parameter $t$ usually interpreted as time. Where a single random variable describes one uncertain quantity, a stochastic process describes an entire trajectory of uncertain quantities unfolding over time, so that the object of study is the random function $t \mapsto X_{t}$ . It matters because almost every real system — stock prices, particle positions, customer queues, population sizes — evolves randomly, and stochastic processes provide the precise mathematical language for prediction, estimation, and control under uncertainty. A surprising fact: Brownian motion, the canonical continuous-time process, has trajectories that are everywhere continuous yet nowhere differentiable — they wiggle infinitely on every scale, which is why ordinary calculus fails and the Itô Calculus was born.

The Core Insight

The entire behavior of a stochastic process is encoded not in any single $X_{t}$ but in how the $X_{t}$ at different times are jointly distributed. Two simplifying structures dominate the subject. Independence/stationarity says the statistical character does not depend on absolute time. The Markov property says the future depends on the past only through the present. Whenever one of these holds, the seemingly infinite-dimensional object collapses into something computable — transition probabilities, rate matrices, or a few moments — and that collapse is what makes the theory usable.

Formal Statement

Stochastic Process

Let $(Ω, F, P)$ be a probability space and $(E, E)$ a measurable state space. A stochastic process with index set $T$ is a family of random variables

X = {X_{t} : t \in T}, X_{t} : Ω \to E .

⟳each

X_{t}

F / E

-measurable; "process" just means the whole indexed family is given on one

(Ω, F, P)

$Ω$ — sample space; an outcome $ω \in Ω$ .
$F$ — $σ$ -algebra of events; $P$ — probability measure.
$E$ — state space (e.g. $R$ , $Z$ , a finite set).
$T$ — index set: $T = N$ (discrete time) or $T = [0, \infty)$ (continuous time).
For fixed $ω$ , the map $t \mapsto X_{t} (ω)$ is a sample path or trajectory.

Finite-Dimensional Distributions

For any finite set of times $t_{1} < \dots < t_{n}$ in $T$ , the finite-dimensional distribution (fdd) is the joint law

μ_{t_{1}, \dots, t_{n}} (A_{1} \times \dots \times A_{n}) = P (X_{t_{1}} \in A_{1}, \dots, X_{t_{n}} \in A_{n}) .

⟳pull back the product event under the map

ω \mapsto (X_{t_{1}} (ω), \dots, X_{t_{n}} (ω))

and apply

P

The fdds must be consistent: marginalizing one coordinate of $μ_{t_{1}, \dots, t_{n}}$ gives the lower-order fdd, and permuting times permutes coordinates accordingly.

Kolmogorov Extension Theorem

Given a consistent family of finite-dimensional distributions ${μ_{t_{1}, \dots, t_{n}}}$ on a Polish state space, there exists a probability space $(Ω, F, P)$ and a stochastic process ${X_{t}}_{t \in T}$ whose fdds are exactly the given family. Thus a process is its consistent fdds.

Markov Property

A process is a Markov process if for all $s < t$ and measurable $A$ ,

P (X_{t} \in A ∣ F_{s}) = P (X_{t} \in A ∣ X_{s}),

where $F_{s} = σ (X_{u} : u \leq s)$ is the history up to time $s$ .

⟳condition on the full past

F_{s}

; the Markov property asserts this equals conditioning on the single present value

X_{s}

Stationarity

A process is (strictly) stationary if shifting time leaves all fdds unchanged: for every shift $h$ and times $t_{1}, \dots, t_{n}$ ,

(X_{t_{1} + h}, \dots, X_{t_{n} + h}) = d (X_{t_{1}}, \dots, X_{t_{n}}) .

It is weakly (covariance) stationary if $E [X_{t}] = μ$ is constant and the autocovariance $γ (s, t) = Cov (X_{s}, X_{t})$ depends only on $∣ t - s ∣$ .

⟳equate the joint distribution of any time-block with that of the same block translated by

h

; weak stationarity only constrains the first two moments.

Why It Holds / Derivation

The defining computational engine of a Markov chain is the Chapman–Kolmogorov equation, which expresses that to travel from state $i$ at time $0$ to state $k$ at time $m + n$ , you must pass through some intermediate state $j$ at the intermediate time.

Chapman–Kolmogorov Equation

Let $X$ be a discrete-time Markov chain on a countable state space with $n$ -step transition probabilities $p_{ij}^{(n)} = P (X_{n} = j ∣ X_{0} = i)$ . We prove

p_{ik}^{(m + n)} = j \sum p_{ij}^{(m)} p_{j k}^{(n)} .

⟳condition on the state at the intermediate time

m

and apply the Markov property to factor the joint probability.

Step 1. Decompose over the intermediate state

X_{m} = j

using total probability:

p_{ik}^{(m + n)} = P (X_{m + n} = k ∣ X_{0} = i) = j \sum P (X_{m + n} = k, X_{m} = j ∣ X_{0} = i) .

Step 2. Factor each term with the multiplication rule:

= j \sum P (X_{m + n} = k ∣ X_{m} = j, X_{0} = i) P (X_{m} = j ∣ X_{0} = i) .

Step 3. Apply the Markov property: given

X_{m} = j

, the future is independent of

X_{0}

, so

P (X_{m + n} = k ∣ X_{m} = j, X_{0} = i) = p_{j k}^{(n)}

. Step 4. Substitute, recognizing

P (X_{m} = j ∣ X_{0} = i) = p_{ij}^{(m)}

p_{ik}^{(m + n)} = j \sum p_{ij}^{(m)} p_{j k}^{(n)} . ■

In matrix form with transition matrix $P = (p_{ij})$ , this is simply $P^{(m + n)} = P^{(m)} P^{(n)}$ , i.e. $P^{(n)} = P^{n}$ — the $n$ -step matrix is the $n$ -th power.

Strong Law for Stationary Ergodic Processes (Birkhoff)

If ${X_{t}}$ is stationary and ergodic with $E ∣ X_{0} ∣ < \infty$ , then the time average converges to the ensemble average almost surely:

\frac{1}{n} k = 0 \sum n - 1 X_{k} a.s. E [X_{0}] .

⟳Birkhoff's pointwise ergodic theorem applied to the shift map; ergodicity forces the invariant

σ

-algebra to be trivial so the limit is the constant mean.

Andrey Kolmogorov | 1903–1987

Kolmogorov gave probability theory its modern measure-theoretic foundation in his 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung, where the extension theorem (Theorem above) showed that consistent fdds always yield a genuine process. He also derived the Kolmogorov forward and backward equations governing the time-evolution of transition densities for Markov processes, the bridge between probability and PDEs. His axioms turned an intuitive but shaky subject into rigorous mathematics, making the entire edifice of stochastic processes possible.

Building Intuition

Picture a particle on a number line. At each tick it flips a fair coin and steps $+ 1$ or $- 1$ : this is the simple symmetric random walk, the discrete ancestor of everything. Now imagine making the time steps tinier and the spatial steps proportionally smaller; in the limit the jagged walk smooths into the continuous, infinitely-wiggly path of Brownian Motion. The Markov property is the statement that the particle "has no memory" — knowing its whole history tells you nothing more about its next step than knowing where it stands right now. Stationarity is the statement that the rules of the game never change, even though the particle's position does. These two ideas — memorylessness and time-homogeneity — are the levers that turn an impossibly rich object into something you can actually compute with.

Three sample paths of a symmetric random walk converging toward a Brownian envelope (dashed parabola $\pm\sqrt{t}$).

Worked Examples

Two-State Markov Chain — Long-Run Weather

Given: Weather is either Sunny (S) or Rainy (R). Transition matrix (rows = today, cols = tomorrow):

P = (0.8 0.4 0.2 0.6) (S,R order) .

Today is Sunny. Find (a) probability it's Rainy in 2 days, (b) the long-run fraction of rainy days. Step 1 (two-step matrix).

P^{2} = (0.8 0.4 0.2 0.6)^{2}

. Top row:

$P_{S S}^{2} = 0.8 (0.8) + 0.2 (0.4) = 0.64 + 0.08 = 0.72$
$P_{S R}^{2} = 0.8 (0.2) + 0.2 (0.6) = 0.16 + 0.12 = 0.28$

So (a) $P (Rainy in 2 days ∣ Sunny today) = 0.28$ .

Step 2 (stationary distribution). Solve

π P = π

π_{S} + π_{R} = 1

. From

π_{S} = 0.8 π_{S} + 0.4 π_{R}

0.2 π_{S} = 0.4 π_{R} \Rightarrow π_{S} = 2 π_{R} .

With

π_{S} + π_{R} = 1

3 π_{R} = 1

, so

π_{R} = 1/3

π_{S} = 2/3

. Answer:

P_{2} (Rain) = 0.28, long-run rainy fraction = \frac{1}{3} \approx 33.3%

Poisson Process — Customer Arrivals

Given: Customers arrive at a counter as a Poisson process with rate

λ = 5

per hour. The number

N_{t}

[0, t]

satisfies

P (N_{t} = k) = e^{- λ t} (λ t)^{k} / k!

, and interarrival times are i.i.d. Exponential(

λ

). Step 1 (probability of exactly 2 in 30 min). Here

t = 0.5

λ t = 5 (0.5) = 2.5

P (N_{0.5} = 2) = e^{- 2.5} \frac{2. 5 ^{2}}{2 !} = e^{- 2.5} \frac{6.25}{2} = 0.0821 \times 3.125 = 0.2565.

Step 2 (expected wait for next customer). Interarrival time

\sim

Exp(

λ

), mean

1/ λ = 1/5

= 12

min. Step 3 (probability next customer takes > 20 min).

P (T > 1/3 h) = e^{- λ t} = e^{- 5 (1/3)} = e^{- 1.667} = 0.189.

Answer:

P (2 in 30 min) \approx 0.257, E [wait] = 12 min, P (wait > 20 min) \approx 0.189

Common Misconceptions

The Markov property is not "no dependence on the past"

Markov processes can be highly correlated across time — yesterday's value strongly influences today's. The property only says that, conditional on the present, the past adds no further predictive information. People also wrongly think every process can be made Markov; in general you need to enlarge the state space (e.g. include velocity, or the last $k$ values) to recover the Markov property.

Time average ≠ ensemble average in general

The equality "long-run time average = mean" (equation [6]) requires ergodicity, not merely stationarity. A stationary process can fail to be ergodic, in which case time averages converge to a random limit depending on which "regime" the system was born into.

A stationary but non-ergodic process

Let $B$ be a single fair coin flip determined once at time zero: $B \in {0, 1}$ each with probability $1/2$ . Define $X_{t} = B$ for all $t$ . This process is strictly stationary (all fdds are shift-invariant). But the time average is $\frac{1}{n} \sum X_{k} = B$ , which equals $0$ or $1$ at random — never the ensemble mean $E [X_{0}] = 1/2$ . Ergodicity fails because the event " $B = 1$ " is shift-invariant yet has probability $1/2$ , not $0$ or $1$ .

Connections

Stochastic processes sit at a crossroads. Discrete-time chains link to Linear Algebra through transition-matrix eigenvalues that govern convergence to equilibrium. The continuous-time limit gives Brownian Motion, whose calculus is the Itô Calculus underpinning Stochastic Differential Equations. The evolution of transition densities is governed by the Kolmogorov Forward Equation (Fokker–Planck), connecting probability to Partial Differential Equations. Conditioning machinery rests on Conditional Expectation and the theory of Martingales, the processes with "fair-game" structure. Stationary processes connect to Fourier Analysis via the spectral density of the autocovariance function.

Real-World Applications

Quantitative finance — the Black–Scholes model treats asset prices as geometric Brownian motion; option pricing reduces to expectations under a martingale measure.
Queueing theory — the M/M/1 queue (Poisson arrivals, exponential service) predicts wait times at call centers, servers, and traffic systems.
Population genetics — the Wright–Fisher and Moran models describe allele-frequency drift as Markov chains; coalescent theory runs them backward in time.
Google PageRank — the web is a giant Markov chain on pages; PageRank is its stationary distribution computed via the power method.
Statistical physics — the Ornstein–Uhlenbeck process models a particle's velocity under friction and thermal noise; Ising-model dynamics use Glauber Markov chains.
Signal processing & control — the Kalman filter estimates a hidden state of a Gauss–Markov process from noisy observations, used in GPS, aerospace, and robotics.
Epidemiology — stochastic SIR models capture extinction and outbreak probabilities that deterministic ODEs miss in small populations.

Historical Development

1827 — Robert Brown observes the erratic jittering of pollen grains in water (Brownian motion), with no mathematical model.
1900 — Louis Bachelier, in Théorie de la spéculation, models stock prices as Brownian motion — five years before Einstein, but largely ignored.
1905 — Albert Einstein gives the physical theory of Brownian motion, linking diffusion to molecular collisions and to Avogadro's number.
1906 — Andrey Markov introduces "Markov chains," analyzing dependent sequences (vowel/consonant patterns in Pushkin's poetry).
1923 — Norbert Wiener constructs Brownian motion rigorously as a measure on path space (the Wiener process).
1931–1933 — Kolmogorov derives the forward/backward equations and, in 1933, axiomatizes probability with the extension theorem.
1940s — Joseph Doob develops martingale theory and writes the foundational Stochastic Processes (1953).
1944–1951 — Kiyoshi Itô develops stochastic integration and Itô's lemma, founding Itô Calculus.
1960 — Rudolf Kálmán publishes the recursive filter for linear stochastic systems, revolutionizing engineering.

Subtleties & Limits

Several deep questions and limitations persist. Path regularity: for a given consistent family of fdds, the Kolmogorov extension theorem guarantees a process but says nothing about continuity of paths; the Kolmogorov continuity theorem gives moment conditions for a continuous modification, but distinguishing versions/modifications is genuinely subtle. Non-Markovian memory: processes like fractional Brownian motion (with Hurst parameter $H \neq = 1/2$ ) exhibit long-range dependence and are not Markov nor semimartingales, breaking standard Itô calculus; building a complete stochastic calculus for them remains active research. High-dimensional mixing rates: for Markov chains used in MCMC, proving sharp bounds on the mixing time (how fast $P^{n}$ reaches stationarity) is open for many practically important chains. Rough paths and SPDEs: giving rigorous meaning to stochastic PDEs driven by extremely irregular noise required Hairer's regularity-structures theory (Fields Medal 2014) and is far from fully understood.