Expectation

Expectation is the computational backbone of statistics, where estimators are evaluated by their expected value (bias) and where maximum likelihood and Bayesian methods both reduce, at some stage, to expectations under a model. In finance, the no-arbitrage pricing of derivatives is built on computing expected discounted payoffs under a risk-neutral probability measure — precisely a LOTUS calculation applied to a payoff function of an underlying asset price. In machine learning, the expected loss (risk) over a data distribution is the quantity that training algorithms attempt to minimize, and LOTUS is used constantly to compute the expected value of a loss function applied to model predictions without needing the (often intractable) distribution of the loss itself. In statistical physics, expectation under the Boltzmann distribution gives thermodynamic averages such as mean energy, connecting directly back to the center-of-mass intuition of this article. In information theory, entropy $H(X) = -E[\log p(X)]$ is itself an expectation of a function of the pmf, a direct LOTUS-style computation. In queueing theory and operations research, expected waiting times and expected costs guide system design, again computed via linearity even when the underlying random variables (arrival times, service times) are heavily dependent. Finally, expectation is the immediate stepping stone to Variance and Moments (which uses $E[(X-E[X])^2]$, itself a LOTUS computation) and to Moment Generating Functions (which package all moments $E[X^n]$ into a single function $E[e^{tX}]$).

:::quiz[If $X$ has $P(X=-2)=0.4$, $P(X=1)=0.5$, $P(X=5)=0.1$, what is $E[X]$?]

• 0.6
• -0.3 ✓
• 1.0
• 2.4

:::quiz[LOTUS lets you compute $E[g(X)]$ using:]

• The distribution of $g(X)$, found first via change of variables
• The distribution of $X$ alone, applied directly with $g$ ✓
• Only the CDF of $g(X)$
• Monte Carlo simulation exclusively

:::quiz[For $X$ with density $f(x)=2x$ on $[0,1]$, what is $E[X^2]$?]

• 1/2
• 1/3
• 2/3
• 1/2... recompute: $\int_0^1 x^2 (2x)\,dx = 1/2$ ✓

:::quiz[Why does the counterexample series fail to define a valid $E[X]$?]

• Because the pmf does not sum to 1
• Because the series converges conditionally but not absolutely, so its sum depends on term order ✓
• Because $X$ takes infinitely many values, which is never allowed
• Because negative values are not permitted for random variables

:::faq[Does linearity extend to infinitely many random variables, $E\big[\sum_{i=1}^\infty X_i\big] = \sum_{i=1}^\infty E[X_i]$?] Not automatically — this requires an extra hypothesis beyond pairwise finiteness of each $E[X_i]$, typically $\sum_i E|X_i| < \infty$, after which the dominated convergence theorem (or monotone convergence, if all $X_i \ge 0$) licenses interchanging the infinite sum and the expectation. Without such a condition, the interchange can fail exactly as in the counterexample above, where infinite rearrangements are not automatically benign. This finite-vs-infinite distinction is why the theorem in this article is stated for two (equivalently, any finite number of) random variables, with the infinite extension treated as a separate, stronger result.

Ask the tutor: Why is independence needed for $E[XY]=E[X]E[Y]$ but not for $E[X+Y]=E[X]+E[Y]$?

Think of the probability mass function of a discrete random variable as a set of point weights placed along the real line: weight $p(x_i)$ sitting at position $x_i$. The expectation $E[X]$ is exactly the center of mass of this system of weights — the balance point of a rigid, weightless rod on which the weights are hung. If you replace weights with a continuous density smeared along the line, the same balance-point interpretation survives, with the sum becoming an integral. This mechanical picture immediately explains several facts that otherwise look mysterious. It explains why $E[X]$ need not equal any value that $X$ can actually take (the center of mass of a two-point system need not coincide with either point). It explains why $E[X]$ is not generally equal to the median or the mode — those are different notions of "typical value," while expectation specifically tracks the balance point. It also foreshadows why $E[X]$ can be pulled outside sums and scalar multiples: shifting or scaling every weight's position shifts or scales the balance point in the same way, and combining two independent physical systems' balance points behaves additively when you consider their sum.

Example 1: A non-symmetric wager with LOTUS. A biased die has $P(X=1) = 0.1$, $P(X=2) = 0.05$, $P(X=3)=0.35$, $P(X=4) = 0.05$, $P(X=5) = 0.15$, $P(X=6) = 0.30$ (these sum to $1$: $0.1+0.05+0.35+0.05+0.15+0.30 = 1.00$). A gambler is paid $g(X) = 10X - X^2$ dollars. To find $E[g(X)]$, LOTUS says we never need the distribution of $g(X)$ itself — only $p_X$ and $g$:

Compute $g(x)$ at each value: $g(1) = 10-1=9$, $g(2)=20-4=16$, $g(3)=30-9=21$, $g(4)=40-16=24$, $g(5)=50-25=25$, $g(6)=60-36=24$. Now weight by the pmf:

Term by term: $0.9 + 0.8 + 7.35 + 1.2 + 3.75 + 7.2$. Summing: $0.9+0.8=1.7$; $1.7+7.35=9.05$; $9.05+1.2=10.25$; $10.25+3.75=14.0$; $14.0+7.2=21.2$. So $E[g(X)] = 21.2$ dollars. As a consistency check via linearity, note $g(X) = 10X - X^2$, so

Compute $E[X] = 1(0.1)+2(0.05)+3(0.35)+4(0.05)+5(0.15)+6(0.30) = 0.1+0.1+1.05+0.2+0.75+1.8 = 4.0$. Compute $E[X^2] = 1(0.1)+4(0.05)+9(0.35)+16(0.05)+25(0.15)+36(0.30) = 0.1+0.2+3.15+0.8+3.75+10.8 = 18.8$. Then $10E[X] - E[X^2] = 40 - 18.8 = 21.2$, matching exactly — confirming both LOTUS and linearity are consistent computational routes to the same number, as the theorem guarantees.

Example 2: A continuous asymmetric density, LOTUS for a non-monotone transformation. Let $X$ have density $f(x) = 3x^2$ on $[0,1]$ (and $0$ elsewhere); check this integrates to $1$: $\int_0^1 3x^2\,dx = [x^3]_0^1 = 1$. Let $Y = g(X) = (2X-1)^2$, a non-monotone (parabolic, folded) transformation of $X$ on $[0,1]$, which is 0 at $x=1/2$ and increases toward both endpoints. Rather than find the distribution of $Y$ (which would require splitting into two branches around $x=1/2$ and adding densities), LOTUS gives the answer directly:

Expand $(2x-1)^2 = 4x^2 - 4x + 1$, so the integrand is

Integrate term by term over $[0,1]$:

So

Thus $E[(2X-1)^2] = 2/5 = 0.4$. Notice how little work this required compared to deriving $f_Y$: LOTUS converted a change-of-variables problem (with a fold at $x=1/2$, requiring $f_Y(y) = f(x_1)|dx_1/dy| + f(x_2)|dx_2/dy|$ for the two preimages) into a single polynomial integral against the known density of $X$.

:::counterexample[Dropping absolute convergence breaks well-definedness of $E[X]$] The hypothesis of absolute convergence (not mere convergence, and not merely "the sum exists in some order") is essential, not a technicality. Consider a discrete random variable $X$ taking values $x_k = (-1)^{k+1}\dfrac{2^k}{k}$ for $k=1,2,3,\dots$ with masses $p(x_k) = 2^{-k}$; these masses sum to $1$ since $\sum_k 2^{-k} = 1$, so this is a legitimate pmf. Now attempt to compute $E[X]$ term by term as ordinarily listed:

which looks like a perfectly good finite answer, $\ln 2 \approx 0.693$. But this series is only conditionally convergent, not absolutely convergent: $\sum_k |x_k| p(x_k) = \sum_k \frac{1}{k} = \infty$ (the harmonic series diverges). By the Riemann rearrangement theorem, a conditionally convergent series can be reordered to converge to any target real number, or to diverge, simply by listing its terms ($+1, -1/2, +1/3,\dots$) in a different order. Since the definition of $E[X]$ as "the sum over all outcomes" makes no reference to an ordering of the countable outcome space (the underlying probability space has no built-in order on its sample points), the "value" $\ln 2$ obtained above is an artifact of one arbitrary enumeration of the $x_k$'s and is not intrinsic to $X$. This is exactly why the definition of $E[X]$ requires absolute convergence: without it, the quantity is not well-defined as a property of the random variable, only of an incidental labeling of its outcomes. This is also why linearity, as proved above, explicitly assumed $E|X|, E|Y| < \infty$: the rearrangement step in the proof is licensed only by absolute convergence, and dropping that hypothesis genuinely breaks the proof, not just its presentation.