The t, Chi-Squared, and F Distributions

The $t$-distribution underlies the ordinary one-sample and two-sample $t$-tests, regression coefficient tests (each coefficient estimate divided by its standard error is exactly $t$-distributed under normal errors), and $t$-based confidence intervals throughout applied statistics, medicine, and engineering quality control. The Chi-Squared distribution governs goodness-of-fit and independence tests in categorical data analysis, likelihood-ratio test asymptotics (Wilks' theorem), and confidence intervals for variances as in Worked Example 2. The $F$-distribution is the backbone of the analysis of variance (ANOVA), nested-model comparison tests in regression (the overall $F$-test and partial $F$-tests), and variance-homogeneity tests preceding pooled $t$-procedures, as in Worked Example 1. In machine learning and statistics more broadly, these exact finite-sample distributions serve as the calibration benchmark against which asymptotic and resampling-based (bootstrap, permutation) methods are checked when normality is doubtful. In physics and signal processing, Chi-Squared statistics quantify goodness-of-fit for measurement models with Gaussian noise, and $F$-tests compare the residual variance of nested physical models. In cryptography and random number testing, Chi-Squared statistics assess whether generated bit streams deviate from a uniform/independent null model.

Ask the tutor: How does Cochran's theorem generalize the independence of $\bar X$ and $S^2$ to the sums of squares used in full ANOVA decompositions?

Think of $k$ independent standard normal "errors" as coordinates of a random point in $\mathbb{R}^k$. The squared Euclidean distance of that point from the origin is exactly a $\chi^2_k$ variable. As $k$ grows, this squared distance concentrates around $k$ (its mean), by the law of large numbers applied to the sum of squares — this is why $\chi^2_k$ shifts rightward and becomes more symmetric as $k$ increases. Student's $t$ arises from a very concrete practical problem. If $X_1,\dots,X_n$ are i.i.d. $N(\mu,\sigma^2)$, the quantity $Z = (\bar X - \mu)/(\sigma/\sqrt n)$ is exactly standard normal, but it requires knowing $\sigma$. Replacing $\sigma$ by the sample standard deviation $S$ introduces extra randomness from the estimation. The resulting pivot $T = (\bar X - \mu)/(S/\sqrt n)$ is no longer normal; it is a standard normal divided by an independent estimate of spread, and this division is precisely what a $t$-distribution formalizes. Because $S$ is itself random and can be small by chance, $T$ has heavier tails than $Z$: extreme values are more likely because an unlucky small $S$ inflates $T$. The $F$-distribution generalizes this comparison idea from "normal vs. estimated spread" to "spread vs. spread": it is the exact distribution of the ratio of two independent variance estimates, each built from a $\chi^2$ scaled by its own degrees of freedom. This is the natural object when comparing the variability of two independent normal samples, or when comparing nested sums of squares in regression and ANOVA.

Explicit densities (for completeness). From the definitions, one can derive by direct change-of-variables:

(Derivation sketch.) The $\chi^2_1$ density follows from $P(Z^2 \le x) = 2\Phi(\sqrt x)-1$, differentiated to give $f(x) = \frac{1}{\sqrt{2\pi x}}e^{-x/2}$, matching the formula with $k=1$ using $\Gamma(1/2)=\sqrt\pi$. The general $\chi^2_k$ density then follows by induction: a $\chi^2_k$ variable is a $\chi^2_{k-1}$ plus an independent $\chi^2_1$, and the sum's density is the convolution of the two, which one can verify by direct integration reproduces the stated Gamma-family form (indeed $\chi^2_k = \mathrm{Gamma}(k/2, 2)$). The $t_\nu$ density is obtained by writing $T = Z/\sqrt{Q/\nu}$, conditioning on $Q=q$ (so $T\mid Q=q \sim N(0,\nu/q)$), and integrating the conditional normal density against the $\chi^2_\nu$ density of $Q$:

which is a Gamma-type integral in $q$ evaluating to the stated closed form. The $F_{m,n}$ density follows analogously from the joint density of $(Q_1,Q_2)$ via the transformation $W = (Q_1/m)/(Q_2/n)$, $U=Q_2$, computing the Jacobian and integrating out $U$.

Example 1 (asymmetric two-sample $t$ and $F$, unequal sizes). A quality engineer measures the tensile strength (in MPa) of steel rods from two production lines. Line A: $n=7$ rods, sample mean $\bar x_A = 402.3$, sample standard deviation $s_A = 5.1$. Line B: $m=11$ rods, sample mean $\bar x_B = 398.7$, sample standard deviation $s_B = 8.4$. Assume both populations are normal with unknown, possibly unequal, variances $\sigma_A^2, \sigma_B^2$, and the two samples are independent. (a) Test whether the variances are equal. Under $H_0: \sigma_A^2 = \sigma_B^2$, Part 4 of the theorem (with roles matched to the larger sample variance on top by convention) gives

since we put the larger-variance sample (B) in the numerator, using $m-1=10$ and $n-1=6$ degrees of freedom respectively. Numerically,

Consulting (or computing) the $F_{10,6}$ distribution, the upper $5\%$ critical value is approximately $4.06$; since $2.712 < 4.06$, we do not reject equality of variances at the $5\%$ level — the observed ratio is compatible with $\sigma_A^2=\sigma_B^2$ even though $s_B$ looks noticeably larger than $s_A$. (b) Test equality of means using a pooled $t$-statistic, justified by the variance test in (a). The pooled variance estimator is

So $S_p = 7.339$. The pooled two-sample statistic is

using the fact (an extension of Part 3, provable by the same independence argument applied to the pooled sum of squares, which is itself $\chi^2_{16}$ by summing two independent $\chi^2_6$ and $\chi^2_{10}$ variables) that the pooled sum of squares over $\sigma^2$ is $\chi^2_{16}$, independent of $\bar X_A - \bar X_B$. Plugging in:

Comparing to $t_{16}$ (two-sided $5\%$ critical value $\approx 2.120$), $|T_{\text{obs}}|=1.015 < 2.120$, so we do not reject $H_0:\mu_A=\mu_B$ at the $5\%$ level: the observed 3.6 MPa mean difference is not statistically significant given the sample sizes and variability.

Example 2 (single-sample $\chi^2$ and $t$ with an odd sample size, $n=9$). A pharmacologist records the potency (mg/mL) of $n=9$ vials from a single batch, assumed $N(\mu,\sigma^2)$:

Step 1 — compute $\bar x$. Sum $= 5.02+4.87+5.15+4.93+5.08+5.21+4.79+5.03+4.96 = 45.04$. So $\bar x = 45.04/9 = 5.00\overline{4} \approx 5.0044$. Step 2 — compute $S^2$. Deviations $(x_i-\bar x)$: $0.0156,\ -0.1344,\ 0.1456,\ -0.0744,\ 0.0756,\ 0.2056,\ -0.2144,\ 0.0256,\ -0.0444$ (each rounded to 4 dp). Squares: $0.000243, 0.018063, 0.021199, 0.005535, 0.005715, 0.042271, 0.045967, 0.000655, 0.001971$. Summing: $0.000243+0.018063=0.018306$; $+0.021199=0.039505$; $+0.005535=0.045040$; $+0.005715=0.050755$; $+0.042271=0.093026$; $+0.045967=0.138993$; $+0.000655=0.139648$; $+0.001971=0.141619$. So $\sum(x_i-\bar x)^2 \approx 0.14162$, and

Step 3 — a $\chi^2$-based confidence interval for $\sigma^2$. By Part 2 of the theorem, $(n-1)S^2/\sigma^2 = 8S^2/\sigma^2 \sim \chi^2_8$. Using $\chi^2_8$ critical values $\chi^2_{8,0.025}=17.535$ and $\chi^2_{8,0.975}=2.180$, a $95\%$ confidence interval for $\sigma^2$ is

Numerically: $0.14162/17.535 = 0.008076$ and $0.14162/2.180 = 0.064963$. So the $95\%$ CI for $\sigma^2$ is approximately $(0.00808,\ 0.06496)$, i.e. for $\sigma$ roughly $(0.0899, 0.2549)$. Step 4 — a $t$-based confidence interval for $\mu$. By Part 3, $(\bar X-\mu)/(S/\sqrt 9) \sim t_8$. The two-sided $95\%$ critical value is $t_{8,0.025}=2.306$. Then

So the $95\%$ confidence interval for $\mu$ is approximately $(4.902,\ 5.107)$ mg/mL. Note that this interval is noticeably wider than the naive $\bar x \pm 1.96\, S/\sqrt n = 5.0044 \pm 0.0869$ one would (incorrectly) get by treating $S$ as if it were the known $\sigma$ and using the normal critical value — the extra width, $0.1023$ vs $0.0869$, is exactly the price of estimating $\sigma$ from only $n=9$ observations, and it is precisely quantified by the $t_8$ distribution rather than $N(0,1)$.