The Tower Property and Conditional Variance

The tower property is the engine behind martingale theory: a process $(M_n)$ is a martingale exactly when $E[M_{n+1}\mid \mathcal F_n]=M_n$, and the nested tower property is what lets us conclude $E[M_n]=E[M_0]$ for all $n$ by repeated conditioning. In Bayesian statistics, the posterior mean and variance of a parameter are computed by exactly this machinery: the prior predictive variance of data decomposes into "variance given the parameter" (sampling noise) plus "variance of the conditional mean" (parameter uncertainty), which is the statistical content of Example 2 above. The Rao–Blackwell theorem in mathematical statistics uses the total variance law directly: if $T$ is a sufficient statistic and $\delta$ an unbiased estimator, then $E[\delta\mid T]$ has variance $\mathrm{Var}(E[\delta\mid T]) \le \mathrm{Var}(\delta)$, because the total variance law forces $\mathrm{Var}(\delta) = \mathrm{Var}(E[\delta\mid T]) + E[\mathrm{Var}(\delta\mid T)]$ with the second term nonnegative. In actuarial science and queueing theory, compound (random-sum) distributions — as in Example 1 — are the standard model for aggregate claims or workload, and the tower/total-variance formulas give the mean and variance of the aggregate without ever computing its full distribution. In machine learning, the bias-variance-type decompositions used to analyze prediction error (e.g., in random-effects or Bayesian model averaging) are direct descendants of the total variance law. In statistical genetics, the partition of phenotypic variance into genetic and environmental components (Fisher's variance components) is again a total-variance-law computation, conditioning on genotype. Finally, this machinery is the direct stepping stone to computing conditional means and variances in the Bivariate Normal Distribution, where $E[X\mid Y]$ turns out to be an exact linear function of $Y$ and $\mathrm{Var}(X\mid Y)$ is a constant (not random) — a strikingly special structural fact that simplifies the total variance law even further, foreshadowing the role of correlation $\rho$ in that setting. The underlying formalism also connects to Markov Chains, where the tower property with respect to the natural filtration encodes the Markov property itself.

:::quiz[If $X$ and $Y$ are independent random variables with $E[X^2]<\infty$, what is $\mathrm{Var}(E[X\mid Y])$?]

• $\mathrm{Var}(X)$
• $E[\mathrm{Var}(X\mid Y)]$
• $0$ ✓
• Undefined without more information

:::quiz[In the law of total variance $\mathrm{Var}(X)=E[\mathrm{Var}(X\mid Y)]+\mathrm{Var}(E[X\mid Y])$, which minimal hypothesis is required?]

• $X$ and $Y$ must be independent
• $E[X^2]<\infty$ ✓
• $Y$ must be discrete
• $X$ and $Y$ must be jointly normal

:::quiz[In the random-sum example ($N\sim$ Geometric($p=0.4$), $X_i\sim$ Exponential($\lambda=0.25$)), what is $E[\mathrm{Var}(S\mid N)]$?]

• $10$
• $16$
• $40$ ✓
• $60$

:::faq[Is $E[X\mid Y]$ a number or a random variable?] It is a random variable — specifically, a function of $Y$, often written $g(Y)$ where $g(y) = E[X\mid Y=y]$. Its randomness comes entirely through $Y$: once $Y$'s value is revealed, $E[X\mid Y]$ becomes the fixed number $g(y)$. This is a common early confusion: $E[X\mid Y=y]$ (a number, for each fixed $y$) and $E[X\mid Y]$ (a random variable, a function of the random $Y$) are related but distinct objects. The tower property and total variance law are statements about the random variable $E[X\mid Y]$, since we take its expectation or variance over the randomness of $Y$.

:::faq[Why does the proof of total variance need the "orthogonality" of the residual $R=X-E[X\mid Y]$?] Variance does not decompose additively for an arbitrary sum $X=A+B$ unless $A$ and $B$ are uncorrelated: in general $\mathrm{Var}(A+B)=\mathrm{Var}(A)+\mathrm{Var}(B)+2\mathrm{Cov}(A,B)$. The whole proof hinges on showing the cross term $\mathrm{Cov}(R, E[X\mid Y])=0$, which follows from the defining property of conditional expectation (specifically, $E[R\mid Y]=0$, proved via "taking out what is known"). Without this orthogonality — which is a genuinely special feature of the conditional-expectation residual, not true for arbitrary decompositions $X=A+B$ — the total variance law would need an extra covariance correction term.

Ask the tutor: How does the law of total variance simplify when $X$ and $Y$ are jointly normal, and why does $\mathrm{Var}(X\mid Y)$ become a constant?

Think of $Y$ as labeling a group or a "state of the world," and $E[X\mid Y=y]$ as the average of $X$ within that group. The tower property says: if you average those within-group averages, weighting by how likely each group is, you get back the overall average — no information is lost or double-counted, because the within-group averages are computed with respect to the same underlying probability measure restricted to each group. The law of total variance is the variance analogue, but it is subtler because variance does not simply average like expectation does. Total spread in $X$ has two independent sources: (1) how spread out $X$ is inside each group — this is $\mathrm{Var}(X\mid Y)$, and we average this over groups since we don't know which group we're in — and (2) how much the group means themselves differ from each other — this is $\mathrm{Var}(E[X\mid Y])$. Even if every group has zero internal spread (each group is a point mass), the group means can differ, contributing all the variance; conversely if all group means coincide, the total variance is entirely within-group. The formula shows these two sources add, with no cross term, because of an orthogonality property of conditional expectation (proved below).

Example 1: A random sum with a geometric number of exponential claims (asymmetric parameters). An insurer receives a random number $N$ of claims in a period, where $N\sim\text{Geometric}(p)$ on $\{1,2,3,\dots\}$ with success probability $p=0.4$, so $P(N=n)=(1-p)^{n-1}p = (0.6)^{n-1}(0.4)$, giving $E[N]=1/p=2.5$ and $\mathrm{Var}(N)=(1-p)/p^2 = 0.6/0.16 = 3.75$. Given $N=n$, the total claim amount is $S = \sum_{i=1}^n X_i$ where $X_1,X_2,\dots$ are i.i.d. exponential with rate $\lambda=0.25$ (mean $4$, variance $16$), independent of $N$. We compute $E[S]$ and $\mathrm{Var}(S)$ using the tower property and total variance law, conditioning on $Y=N$. Conditional mean and variance given $N=n$. Since the $X_i$ are i.i.d. with mean $\mu=4$ and variance $\sigma^2=16$, and $S=\sum_{i=1}^n X_i$ is a sum of a fixed number $n$ of i.i.d. terms once we condition on $N=n$,

So as random variables in $N$:

Applying the tower property.

Applying the law of total variance. We need both pieces:

Therefore

So $E[S]=10$ and $\mathrm{Var}(S)=100$ (a standard deviation of $10$). Notice the two contributions are comparable in size (40 vs 60): the "between-group" variability — driven by not knowing how many claims there will be — actually dominates slightly here, illustrating that randomness in the count $N$ can be as important a source of overall variance as randomness in the individual claim sizes. This is the compound-distribution / random-sum formula

recovered here as a direct instance of the general theorem, with $E[N]\mathrm{Var}(X) = 2.5(16)=40$ and $\mathrm{Var}(N)(E[X])^2 = 3.75(16) = 60$, matching exactly.

Example 2: Two-stage sampling — a mixture with different within-group shapes (Beta-Binomial style setup). A factory has machines whose defect probability $\Theta$ varies from batch to batch: $\Theta\sim\text{Beta}(\alpha,\beta)$ with $\alpha=2,\ \beta=3$ (so most batches have relatively low defect rates, but with a right skew allowing worse batches). Recall for $\Theta\sim\mathrm{Beta}(\alpha,\beta)$,

Given $\Theta=\theta$, we inspect $n=10$ items from that batch, and $X$ (the number defective) is $\mathrm{Binomial}(10,\theta)$ conditionally. This is the Beta-Binomial model. We find $E[X]$ and $\mathrm{Var}(X)$. Conditional moments. For $X\mid \Theta=\theta \sim \text{Binomial}(10,\theta)$,

Tower property.

Total variance — between-group term.

Total variance — within-group term. We need $E[\Theta(1-\Theta)] = E[\Theta]-E[\Theta^2]$. Since $\mathrm{Var}(\Theta)=E[\Theta^2]-(E[\Theta])^2$, we get $E[\Theta^2] = 0.04 + 0.16 = 0.20$. Hence

and so

Combine.

So $E[X]=4$, $\mathrm{Var}(X)=6$. As a check against the closed-form Beta-Binomial variance formula

which matches exactly. Notice here the between-batch variance ($4.0$) exceeds the within-batch binomial variance ($2.0$): heterogeneity across batches is again the dominant source of overall variability, a common pattern in hierarchical/random-effects data (this is sometimes called "overdispersion" relative to a plain Binomial$(10, 0.4)$, whose variance would only be $10(0.4)(0.6)=2.4$).