Interpreting p-values
A p-value is one of the most important — and most misunderstood — numbers in statistics. When you conduct a hypothesis test, you start by assuming the null hypothesis is true (for example, that a coin is fair, or that a new drug has no effect). The p-value then tells you: if the null hypothesis really were true, how likely would it be to observe data at least as extreme as what you actually got? A small p-value means your data would be very surprising under the null hypothesis, which is evidence against it. For inference about proportions, the null hypothesis usually takes the form H₀: p = p₀, where p₀ is some claimed population proportion. After calculating a test statistic from your sample, you convert it to a p-value using the standard normal distribution. The p-value is a probability — it always falls between 0 and 1 — but it is NOT the probability that the null hypothesis is true. This distinction is critical and is the source of enormous confusion even among professionals. Statisticians use a significance level α (alpha), most commonly 0.05, as a threshold. If p ≤ α, we reject the null hypothesis and call the result statistically significant. If p > α, we fail to reject the null hypothesis. Failing to reject does not mean the null hypothesis is proven true; it only means the data did not provide sufficient evidence against it. Understanding p-values correctly is essential for reading scientific studies, evaluating claims, and making data-driven decisions.
- State the null and alternative hypotheses
- Calculate the test statistic z
- Find the p-value using the normal distribution
- Compare p-value to α and interpret in context
- Understand what the p-value is NOT
State the null and alternative hypotheses
The null hypothesis H₀ specifies a particular value for the population proportion, e.g. H₀: p = 0.50. The alternative hypothesis Hₐ states what you suspect instead: Hₐ: p ≠ 0.50 (two-tailed), Hₐ: p > 0.50 (right-tailed), or Hₐ: p < 0.50 (left-tailed). The direction of Hₐ determines how you calculate the p-value from the test statistic.
Calculate the test statistic z
For a one-proportion z-test, compute z = (p̂ − p₀) / √(p₀(1−p₀)/n), where p̂ is your sample proportion, p₀ is the null value, and n is the sample size. This z-score measures how many standard errors your sample proportion sits away from the null-hypothesized value. Large |z| values indicate the sample result is far from what was expected under H₀.
Find the p-value using the normal distribution
For a right-tailed test: p-value = P(Z ≥ z) = 1 − Φ(z). For a left-tailed test: p-value = P(Z ≤ z) = Φ(z). For a two-tailed test: p-value = 2 · P(Z ≥ |z|) = 2(1 − Φ(|z|)). Here Φ denotes the cumulative standard normal distribution function. In practice you use a z-table or calculator. The p-value is the area in the tail(s) of the normal curve beyond your observed z.
Compare p-value to α and interpret in context
If p ≤ α, reject H₀. State: 'There is sufficient evidence at the α significance level to conclude [alternative claim in context].' If p > α, fail to reject H₀. State: 'There is not sufficient evidence at the α significance level to conclude [alternative claim].' Never say 'we accept H₀' or 'we proved H₀ is true.' Always interpret your conclusion in the real-world context of the problem, not just in abstract statistical language.
Understand what the p-value is NOT
The p-value is NOT: the probability that H₀ is true; the probability that your conclusion is wrong; or the probability that the results occurred by chance in general. It IS: the probability of obtaining a test statistic as extreme or more extreme than the one observed, given that H₀ is true. Think of it as measuring how compatible your data are with the null hypothesis — the smaller the p-value, the less compatible.
The Key Equation
- \hat{p} — Sample proportion (dimensionless): The proportion of successes observed in the sample, computed as x/n.
- p_0 — Null proportion (dimensionless): The specific population proportion claimed by the null hypothesis.
- n — Sample size (count): The number of individuals in the sample.
- \Phi — Standard normal CDF (dimensionless): The cumulative distribution function of the standard normal distribution; Φ(z) = P(Z ≤ z).
- z_{\text{obs}} — Observed z-score (standard errors): The number of standard errors the sample proportion lies above the null value.
Worked Example
Problem. A school claims that 60% of its students walk to school. A student reporter surveys 120 randomly selected students and finds that 63 of them walk. The reporter suspects the true proportion is less than 60%. Test at α = 0.05. Calculate the p-value and interpret it.
Answer. p-value ≈ 0.0468 < 0.05; reject H₀. There is sufficient evidence that fewer than 60% of students walk to school.
Origins · 1 minRonald A. Fisher and how this was discovered
Ronald Fisher introduced the concept of the p-value and significance testing in the 1920s, most notably in his landmark 1925 book 'Statistical Methods for Research Workers.' He proposed p < 0.05 as a convenient threshold for 'significance,' a convention that persists — and is debated — to this day. Fisher's framework of null hypothesis significance testing (NHST) became the dominant statistical paradigm in biology, psychology, and medicine throughout the 20th century. Fisher was a passionate gardener and developed many of his statistical ideas while working at the Rothamsted agricultural research station in England, where he needed methods to analyse messy crop-yield data. He also made major contributions to genetics, evolutionary biology, and was a leading figure in the modern evolutionary synthesis.
Related Concepts
- One-Proportion z-Test — The one-proportion z-test is the procedure that generates the z-statistic from which the p-value for proportions is calculated.
- Null and Alternative Hypotheses — The p-value is defined relative to the null hypothesis, so understanding how to state H₀ and Hₐ is a prerequisite for interpreting p-values.
- Significance Level (α) — The significance level is the threshold against which you compare the p-value to make a reject/fail-to-reject decision.
- Type I and Type II Errors — The p-value threshold α equals the Type I error rate — the probability of rejecting a true null hypothesis — connecting p-values directly to the error framework of hypothesis testing.
- Confidence Intervals for Proportions — Confidence intervals and p-values are complementary tools: a 95% confidence interval excludes p₀ exactly when the two-tailed p-value is less than 0.05.