Inference for Means

Inference for means uses sample data to draw conclusions about an unknown population mean μ. The key tool is the t-distribution, which accounts for the extra uncertainty of estimating the standard deviation from the same sample. Applies whenever you want to estimate or test a claim about an average — heights, test scores, drug effects, manufacturing tolerances. Foundational work by W.S. Gosset ("Student") in 1908.

Inference for means is the branch of statistical inference concerned with the unknown population mean μ of a quantitative variable, using a random sample of size n. We construct confidence intervals to estimate μ and perform hypothesis tests to evaluate claims about μ. Because the population standard deviation σ is almost never known in practice, we estimate it with the sample standard deviation s, which introduces additional variability; the correct sampling distribution is then Student's t-distribution with n − 1 degrees of freedom, not the normal distribution. This matters enormously: using the normal curve with estimated σ gives intervals that are too narrow and tests that reject the null hypothesis too often. The surprising fact is that even though the t-distribution was derived for small samples, the resulting procedures are remarkably robust to violations of the normality assumption when n is moderately large.

The Core Insight

When σ is known, the standardized sample mean follows a standard normal distribution. When σ is unknown and replaced by s, the denominator becomes random, fattening the tails — the result is the t-distribution. Every confidence interval and test for a mean has the same skeleton: estimate ± (critical value) × (standard error), where the standard error of the mean is $s / n$ .

Formal Statement

Sampling Distribution of the Sample Mean

Let $X_{1}, X_{2}, \dots, X_{n}$ be a random sample from a population with mean μ and standard deviation σ. The sample mean is

\overset{ˉ}{X} = \frac{1}{n} i = 1 \sum n X_{i}

⟳The sample mean is simply the arithmetic average of the observations — the sum divided by the count.

Its expected value and standard deviation (the standard error) are

E [\overset{ˉ}{X}] = μ, SE (\overset{ˉ}{X}) = \frac{σ}{n}

⟳Use linearity of expectation for the mean; for the SE, use

Var (\overset{ˉ}{X}) = \frac{1}{n ^{2}} \sum Var (X_{i}) = σ^{2} / n

assuming independence, then take the square root.

Symbols:

\overset{ˉ}{X}

= sample mean (random variable);

X_{i}

= the i-th observation; n = sample size; μ = population mean; σ = population standard deviation; SE = standard error.

The t-Statistic

If the population is normally distributed (or n is large), then when σ is unknown and estimated by the sample standard deviation s, the standardized statistic

t = \frac{X ˉ - μ}{s / n}

⟳Replace σ in the z-score by the estimator s; the ratio of a standard normal to an independent scaled chi random variable defines the t-distribution.

follows Student's t-distribution with $df = n - 1$ degrees of freedom, where the sample standard deviation is

s = \frac{1}{n - 1} i = 1 \sum n (X_{i} - \overset{ˉ}{X})^{2}

⟳This is the unbiased estimator of σ²; dividing by n−1 (Bessel's correction) compensates for using

\overset{ˉ}{X}

in place of the true μ, which "uses up" one degree of freedom.

Confidence Interval for a Mean

A level C confidence interval for μ (when σ is unknown) is

\overset{x}{ˉ} \pm t_{n - 1}^{*} \frac{s}{n}

⟳Invert the probability statement

P (- t^{*} < t < t^{*}) = C

and solve the inequality for μ, treating

\overset{x}{ˉ}

and s as fixed.

where $t_{n - 1}^{*}$ is the critical value cutting off area $(1 - C) /2$ in each tail of the t-distribution with n − 1 degrees of freedom. The quantity $t^{*} \cdot s / n$ is the margin of error.

One-Sample t-Test

To test the null hypothesis $H_{0} : μ = μ_{0}$ , compute the test statistic

t = \frac{x ˉ - μ _{0}}{s / n}, df = n - 1

⟳This is the t-statistic of equation [3] evaluated under the assumption that the true mean equals the hypothesized value

μ_{0}

The p-value is the probability, computed from the $t_{n - 1}$ distribution, of obtaining a statistic at least as extreme as the observed t in the direction(s) specified by the alternative hypothesis $H_{a}$ .

Why It Holds / Derivation

Why the t-Distribution Replaces the Normal

Step 1. Start from the known-σ case. If the population is normal, then

\overset{ˉ}{X} \sim N (μ, σ^{2} / n)

exactly, so the z-score

Z = (\overset{ˉ}{X} - μ) / (σ / n)

is standard normal

N (0, 1)

. Step 2. In practice σ is unknown. We substitute the random estimator s. Now the denominator

s / n

varies from sample to sample — sometimes s underestimates σ, producing unusually large statistics. Step 3. This extra variability fattens the tails: the statistic

t = (\overset{ˉ}{X} - μ) / (s / n)

is more likely to land far from zero than a normal variable would be. Step 4. Formally,

t

can be written as

Z / V / (n - 1)

, where

Z \sim N (0, 1)

and

V = (n - 1) s^{2} / σ^{2} \sim χ_{n - 1}^{2}

are independent. This ratio is precisely the definition of a t-distribution with n − 1 degrees of freedom. Step 5. As n → ∞, s converges to σ (law of large numbers), the chi-square term concentrates at 1, and the t-distribution converges to the standard normal. Hence for large n the two procedures nearly coincide. ∎

William Sealy Gosset ("Student") | 1876–1937

Gosset was a chemist and mathematician employed by the Guinness brewery in Dublin. Working with small samples of barley, hops, and yeast, he needed a way to make reliable inferences when only a handful of measurements were available — exactly the situation where the normal approximation fails. He derived the t-distribution in 1908. Because Guinness forbade employees from publishing (to protect trade secrets), he wrote under the pseudonym "Student", and the distribution has carried that name ever since. R.A. Fisher later supplied the rigorous mathematical foundation and the geometric interpretation in terms of degrees of freedom.

Building Intuition

Think of estimating μ as throwing a ring (the confidence interval) at a fixed peg (the true mean). The ring is centered at your sample mean $\overset{x}{ˉ}$ , and its radius is the margin of error. With a 95% procedure, 95% of all possible rings you might throw would land over the peg — but for any single ring you don't know whether yours is one of them.

Why is the t-ring wider than the z-ring? Because you also had to guess the radius from the same throw. When you estimate σ from a small sample, you're uncertain about how wide to make the ring, and the only safe response is to make it a bit wider to compensate. The fewer observations you have, the less you trust s, the more extra width the t-critical value adds — at $df = 1$ , $t^{*} \approx 12.7$ for 95%, but it shrinks toward $z^{*} = 1.96$ as n grows.

Standard normal (z) versus t-distribution: the t-curve has heavier tails

<svg viewBox="0 0 480 300" xmlns="http://www.w3.org/2000/svg">
<rect width="480" height="300" fill="#0a0e17"/>
<line x1="40" y1="250" x2="450" y2="250" stroke="#94a3b8" stroke-width="1.5"/>
<line x1="245" y1="60" x2="245" y2="250" stroke="#94a3b8" stroke-width="1" stroke-dasharray="3,3"/>
<text x="245" y="270" fill="#e2e8f0" font-size="11" text-anchor="middle">0</text>
<text x="130" y="270" fill="#e2e8f0" font-size="11" text-anchor="middle">-2</text>
<text x="360" y="270" fill="#e2e8f0" font-size="11" text-anchor="middle">+2</text>
<!-- normal curve (light) -->
<path d="M40,249 C120,247 180,150 245,75 C310,150 370,247 450,249" fill="none" stroke="#94a3b8" stroke-width="1.5"/>
<!-- t curve (gold), lower peak, fatter tails -->
<path d="M40,243 C120,235 185,160 245,95 C305,160 370,235 450,243" fill="none" stroke="#d4a853" stroke-width="2.5"/>
<text x="300" y="100" fill="#d4a853" font-size="12">t (df small)</text>
<text x="300" y="70" fill="#94a3b8" font-size="12">z (normal)</text>
<!-- tail emphasis -->
<text x="395" y="235" fill="#d4a853" font-size="10">heavier tails</text>
<line x1="390" y1="240" x2="430" y2="247" stroke="#d4a853" stroke-width="1"/>
<text x="100" y="290" fill="#e2e8f0" font-size="10">t-curve: lower center, thicker tails → wider intervals</text>
</svg>

Worked Examples

Confidence Interval for Mean Battery Life

Given: A sample of n = 16 batteries has mean lifetime

\overset{x}{ˉ} = 42.0

hours and sample standard deviation

s = 4.8

hours. Find a 95% confidence interval for the true mean lifetime μ. Assume battery lives are approximately normal. Step 1 — Degrees of freedom:

df = n - 1 = 15

. Step 2 — Critical value: For 95% confidence with

df = 15

t^{*} = 2.131

(from a t-table). Step 3 — Standard error:

SE = \frac{s}{n} = \frac{4.8}{16} = \frac{4.8}{4} = 1.20 hours

Step 4 — Margin of error:

ME = t^{*} \cdot SE = 2.131 \times 1.20 = 2.557 hours

Step 5 — Interval:

42.0 \pm 2.557 \Rightarrow (39.44, 44.56)

Answer: We are 95% confident the true mean battery life is between 39.44 and 44.56 hours.

One-Sample t-Test on a Quality Claim

Given: A cereal box is supposed to contain μ₀ = 350 g. A regulator samples n = 25 boxes and finds

\overset{x}{ˉ} = 347.2

g with

s = 6.0

g. At the 5% significance level, is there evidence the mean fill is below 350 g? Step 1 — Hypotheses:

H_{0} : μ = 350

vs.

H_{a} : μ < 350

(one-sided). Step 2 — Test statistic:

t = \frac{x ˉ - μ _{0}}{s / n} = \frac{347.2 - 350}{6.0/ 25} = \frac{- 2.8}{1.20} = - 2.333

Step 3 — Degrees of freedom:

df = 24

. Step 4 — p-value: For

t = - 2.333

with

df = 24

(left tail),

p \approx 0.014

. Step 5 — Decision: Since

p = 0.014 < 0.05

, reject $H_{0}$ .

Answer: There is statistically significant evidence ( $t = - 2.33$ , $p \approx 0.014$ ) that the mean fill is less than 350 g. The manufacturer appears to be underfilling.

Common Misconceptions

Confidence Is About the Procedure, Not the Interval

A 95% confidence interval does not mean "there is a 95% probability that μ lies in this interval." Once computed, the interval $(39.44, 44.56)$ either contains μ or it does not — there is no probability left. The 95% refers to the long-run success rate of the method: if you repeated the sampling many times, about 95% of the intervals produced would capture μ. Similarly, a p-value is not the probability that $H_{0}$ is true; it is the probability of data at least this extreme assuming $H_{0}$ is true.

When the t-Procedure Misleads

The t-procedure assumes the sampling distribution of $\overset{ˉ}{X}$ is approximately normal. Suppose you sample n = 8 incomes from a population with a few extreme billionaires — a strongly right-skewed distribution with outliers. With such a small n, the Central Limit Theorem has not "kicked in," and the single largest value can dominate both $\overset{x}{ˉ}$ and s. The computed 95% interval may capture the true mean far less than 95% of the time, and a t-test's actual error rate can be much higher than the nominal 5%. The remedy: larger n, transforming the data, or using a method designed for skewed data.

Connections

Inference for means sits inside a web of related ideas. The validity of the sample-mean's normality rests on the Central Limit Theorem, which guarantees approximate normality of $\overset{ˉ}{X}$ for large n regardless of population shape. When σ is known, the procedures use the Normal Distribution and a z-statistic instead of the t. Comparing two group means extends to the Two-Sample t-Test and paired t-tests, while comparing three or more means generalizes to Analysis of Variance. The complementary problem of estimating proportions is handled by Inference for Proportions, which has the same "estimate ± margin of error" skeleton but a different standard error. The general logic of $H_{0}$ , $H_{a}$ , and p-values comes from Hypothesis Testing.

Real-World Applications

Pharmaceutical trials — estimating the mean reduction in blood pressure produced by a new drug, with a confidence interval reported to regulators (FDA, EMA).
Manufacturing quality control — testing whether the mean diameter of machined parts meets specification, as in the cereal-box example above.
Agriculture — Gosset's original use: comparing mean crop yields under different fertilizers or barley strains.
Education research — estimating the mean improvement in standardized test scores after an intervention.
Environmental monitoring — testing whether the mean concentration of a pollutant in a river exceeds a legal limit.
Clinical chemistry — establishing reference ranges for blood-test values (e.g., mean cholesterol) using sample data.
Polling and surveys — reporting the average household spending with a margin of error.

Historical Development

1733 — Abraham de Moivre derives the normal curve as an approximation to the binomial, foundation for later sampling theory.
1809–1810 — Gauss and Laplace develop the normal distribution and the method of least squares; Laplace proves an early Central Limit Theorem.
1877 — Friedrich Helmert derives the chi-square distribution of the sample variance, a key ingredient of the future t-distribution.
1908 — W.S. Gosset, writing as "Student," publishes The Probable Error of a Mean in Biometrika, introducing the t-distribution for small samples.
1912–1925 — R.A. Fisher provides a rigorous derivation, coins the term "degrees of freedom," and gives the geometric interpretation.
1925 — Fisher's Statistical Methods for Research Workers popularizes the t-test and tabulates critical values for practitioners.
1933 — Jerzy Neyman and Egon Pearson formalize the theory of hypothesis testing (errors, power), framing the modern decision structure.
1937 — Neyman develops the formal theory of confidence intervals, giving the modern interpretation used today.
Late 20th century — Computers make exact t-computations and simulation-based robustness studies routine, refining guidance on when normality assumptions matter.

Subtleties & Limits

Several practical questions resist clean answers. How large must n be for the t-procedure to be trustworthy with non-normal data? The classic "n ≥ 30" rule is a crude heuristic — for symmetric distributions even small n works, while for strongly skewed data even n = 50 may be inadequate. Outliers can wreck both $\overset{x}{ˉ}$ and s; robust alternatives (trimmed means, the bootstrap, rank-based tests) trade some efficiency for resistance. There is ongoing debate over the interpretation and misuse of p-values, culminating in the American Statistical Association's 2016 statement warning against treating $p < 0.05$ as automatic proof. Finally, choosing between frequentist confidence intervals and Bayesian credible intervals remains a genuine philosophical and practical fork: the latter can directly state "there is a 95% probability μ lies in this interval," but only after committing to a prior distribution.