The Bivariate Normal Distribution

The bivariate (and general multivariate) normal is the workhorse behind linear regression: the conditional mean formula proved above, $E[Y\mid X=x] = \mu_Y + \rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X)$, is exactly the population regression line, and ordinary least squares recovers its sample analogue. In finance, joint normality of asset log-returns underlies the Markowitz mean–variance portfolio theory and the covariance matrix used in risk models. In signal processing and communications, additive Gaussian noise models routinely require the joint (not just marginal) normality assumption to compute error probabilities like the one in Worked Example 1. In machine learning and statistics, the bivariate normal is the $2$-dimensional case of the multivariate Gaussian used in Gaussian discriminant analysis, Kalman filtering, and Gaussian process regression, all of which rely on the same completing-the-square argument to derive closed-form conditional distributions. In physics, jointly Gaussian random fields (e.g. in statistical mechanics and cosmology, describing correlated fluctuations) use the identical machinery. Finally, this two-variable case is the seed of the Central Limit Theorem: sums of many small, weakly dependent contributions tend toward jointly Gaussian vectors, and understanding the two-dimensional conditional/independence structure here is the natural stepping stone to the general multivariate Gaussian limit theory.

:::quiz[For jointly normal $(X,Y)$ with $\rho = -0.6$, $\sigma_X=4$, $\sigma_Y=5$, the slope of $E[Y\mid X=x]$ in $x$ is]

• $-0.6$
• $-0.75$ ✓
• $0.75$
• $-3.0$

Ask the tutor: How does this bivariate picture extend to the general multivariate normal distribution and its role in the multivariate Central Limit Theorem?

Picture the density surface: it is a single smooth bump over the plane, elliptical in cross-section. Slicing the surface horizontally at any height produces an ellipse; these ellipses are all similar and concentric, centered at $(\mu_X,\mu_Y)$, and their axes are tilted relative to the coordinate axes whenever $\rho \ne 0$. The tilt is the visual signature of dependence: if $\rho>0$, the ellipses lean so that large $X$ tends to accompany large $Y$; if $\rho<0$, they lean the other way; if $\rho=0$, the ellipses are axis-aligned (circles when $\sigma_X=\sigma_Y$), and the surface actually factors into a product of two independent univariate bumps. The conditional distribution of $Y$ given $X=x$ can be visualized as taking a vertical slice through the bump at a fixed $x$. Because of the special algebraic structure of the exponent (it is a quadratic form in $(x,y)$), this slice — suitably rescaled — is again a normal density in $y$, whose center drifts linearly in $x$ along the famous regression line $y = \mu_Y + \rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X)$, and whose spread $\sigma_Y^2(1-\rho^2)$ shrinks toward zero as $|\rho|\to 1$ (the slice becomes a spike) and stays at $\sigma_Y^2$ when $\rho=0$ (slicing does not narrow the spread at all, consistent with independence).

The technique throughout is completing the square in the joint exponent to force a factorization $(\ast)$, combined with the tower property of conditional expectation for part (3); no result depending on this theorem was used, so the proof is non-circular.

Example 1 (Asymmetric parameters — computing a conditional probability). Two components of a signal, $X$ (in volts) and $Y$ (in volts), are jointly normal with

So $\sigma_X = 3$, $\sigma_Y = 2$. Suppose we observe $X = 5$. Find $P(Y > 0 \mid X=5)$. By part (2) of the theorem,

Compute the conditional mean:

so the conditional mean is $-1 + (-1.2) = -2.2$. Compute the conditional variance:

so the conditional standard deviation is $\sqrt{2.56} = 1.6$. Thus $Y\mid X=5 \sim N(-2.2,\, 2.56)$, and

where $Z$ is standard normal. From standard normal tables, $P(Z>1.375) \approx 0.0846$. Notice this is much smaller than the unconditional probability $P(Y>0) = P(Z > (0-(-1))/2) = P(Z>0.5)\approx 0.3085$: observing an unusually high $X$, combined with the negative correlation, pulls the predicted $Y$ down substantially, making $Y>0$ far less likely.

Example 2 (Verifying the correlation from a linear model, and checking consistency). Suppose test scores satisfy $X \sim N(70, 100)$ (mean 70, variance 100, so $\sigma_X=10$) and

where $Y$ represents a second test score. We claim $(X,Y)$ is jointly (bivariate) normal, and we compute all five parameters. Joint normality. For any constants $a,b$ not both zero,

Since $X$ and $\varepsilon$ are independent normal (or degenerate) random variables, any linear combination $(a+0.8b)X + b\varepsilon$ of independent normals is again normal (a fact from Sums of Random Variables and Convolution: the sum of independent normals is normal, and a constant multiple of a normal is normal), and adding the constant $15b$ preserves normality. Hence $aX+bY$ is normal for every $a,b$, so by the general definition, $(X,Y)$ is jointly normal. Parameters. $\mu_X = 70$, $\sigma_X^2=100$. For $Y$:

using independence of $X,\varepsilon$ to add variances. So $\sigma_Y = 10$. For the covariance,

so

Consistency check via the theorem. The theorem's formula for the conditional mean predicts

which exactly matches $E[Y\mid X=x] = 15+0.8x+E[\varepsilon]=15+0.8x$ computed directly from the linear model — confirming the theorem's regression-line formula against an independent derivation. Likewise the theorem predicts conditional variance $\sigma_Y^2(1-\rho^2) = 100(1-0.64)=36$, exactly $\mathrm{Var}(\varepsilon)$, as it must be since $\varepsilon$ is precisely the "noise" left in $Y$ after regressing on $X$.