Experiments and Causal Inference

In statistics, one of the most powerful goals is not just to describe data or find patterns, but to determine whether one variable actually causes a change in another. Causal inference is the process of drawing valid conclusions about cause-and-effect relationships from data. While observational studies can reveal correlations, only a well-designed experiment — one that systematically manipulates a variable while controlling all others — gives us the logical foundation to claim causation. The key principle is randomization. When researchers randomly assign subjects to treatment and control groups, they neutralize the effect of confounding variables — hidden factors that might otherwise create a misleading appearance of causation. For example, suppose students who sleep more also score higher on tests. Does sleep cause better scores, or do more motivated students both study harder and sleep earlier? An observational study cannot untangle this. A randomized controlled experiment, however, can isolate sleep as the cause by randomly assigning sleep durations. Understanding experiments and causal inference equips you to critically evaluate scientific claims, medical studies, and policy decisions. Claims like 'Coffee reduces cancer risk' or 'Social media causes depression' require careful scrutiny: Was the study experimental or observational? Were subjects randomly assigned? Were confounders controlled? This article develops the framework that answers these questions rigorously.

Key takeaways
  • Identifying the Explanatory and Response Variables
  • The Role of the Control Group and Placebo
  • Randomization and Its Mathematical Justification
  • Replication and Statistical Significance
  • Blinding: Single and Double
  • Internal vs. External Validity

Identifying the Explanatory and Response Variables

Every experiment has an explanatory variable (also called the independent variable or factor), which the researcher deliberately manipulates, and a response variable (dependent variable), which is measured as the outcome. For causal claims, you must specify exactly which variable you believe causes changes in the other. Example: 'Does the amount of fertilizer (explanatory) affect plant height (response)?'

The Role of the Control Group and Placebo

A control group receives no treatment (or a placebo — an inert substitute that mimics the treatment in appearance). The control group establishes a baseline, so any difference in the response variable can be attributed to the treatment, not to the mere act of being in an experiment (the placebo effect). Without a control group, there is no valid comparison and no causal claim.

Randomization and Its Mathematical Justification

Random assignment ensures each subject has an equal probability of being placed in any group. Mathematically, if you have n subjects and assign them to two equal groups of n/2 by simple random sampling, the expected value of any confounding variable C is equal across groups: E[C | treatment] = E[C | control]. This balancing of confounders — both measured and unmeasured — is what makes causal inference legitimate. No observational design can guarantee this.

Replication and Statistical Significance

Replication means applying each treatment to multiple experimental units, not just one. With more replicates, random variation averages out and true treatment effects become visible. The average treatment effect (ATE) is estimated as the difference in group means: ATE = ȳ_treatment − ȳ_control. A large ATE relative to sampling variability (measured by standard error) suggests the effect is real, not due to chance. This is evaluated using hypothesis testing.

Blinding: Single and Double

Blinding prevents bias from expectation. In a single-blind experiment, subjects do not know which group they are in. In a double-blind experiment, neither subjects nor evaluators know group assignments until analysis is complete. Double-blinding eliminates both the placebo effect on subjects and unconscious bias by researchers who record outcomes. These controls are essential for valid causal conclusions in medical and behavioral research.

Internal vs. External Validity

Internal validity refers to whether the experiment correctly identifies a causal relationship within the study — achieved through randomization, control, and blinding. External validity (generalizability) refers to whether conclusions apply beyond the specific sample and setting. A tightly controlled lab experiment may have high internal validity but low external validity if its conditions differ dramatically from the real world. Both must be considered when evaluating causal claims.

The Key Equation

τ^=Yˉtreatment−Yˉcontrol=1nT∑i∈TYi−1nC∑j∈CYj\hat{\tau} = \bar{Y}_{\text{treatment}} - \bar{Y}_{\text{control}} = \frac{1}{n_T}\sum_{i \in T} Y_i - \frac{1}{n_C}\sum_{j \in C} Y_j
  • \hat{\tau} — Estimated Average Treatment Effect (same units as Y): The estimated causal effect of the treatment; positive means the treatment increased the response on average.
  • \bar{Y}_{\text{treatment}} — Treatment group mean (same units as Y): The average value of the response variable among subjects who received the treatment.
  • \bar{Y}_{\text{control}} — Control group mean (same units as Y): The average value of the response variable among subjects in the control group, serving as the baseline.
  • n_T, n_C — Group sizes (subjects): The number of experimental units in the treatment and control groups respectively.

Worked Example

Problem. A school psychologist tests whether a new study-skills workshop improves exam scores. She randomly assigns 60 students: 30 attend the workshop (treatment) and 30 do not (control). After the next exam, the treatment group's mean score is 78.4 with standard deviation 9.2, and the control group's mean score is 72.1 with standard deviation 10.5. (a) Calculate the estimated average treatment effect. (b) Calculate the standard error of the difference in means. (c) Determine whether this provides evidence of a causal effect.

τ^=6.3 points,SE=9.2230+10.5230≈2.55,t≈2.47,p≈0.016\hat{\tau} = 6.3 \text{ points}, \quad SE = \sqrt{\frac{9.2^2}{30}+\frac{10.5^2}{30}} \approx 2.55, \quad t \approx 2.47, \quad p \approx 0.016

Answer. The workshop caused an estimated improvement of 6.3 points (SE ≈ 2.55, t ≈ 2.47, p ≈ 0.016). Because randomization was used, this is a valid causal conclusion.

Figure
Diagram
Randomized Controlled Experiment FlowSample of 60 SubjectsRandomAssignmentTreatment Groupn = 30 (workshop)Control Groupn = 30 (no workshop)ȳ_T = 78.4Measure exam scoresȳ_C = 72.1Measure exam scoresATE = 78.4 − 72.1= 6.3 points (causal)
Origins · 1 minRonald A. Fisher and how this was discovered
RA
Ronald A. Fisher
1890–1962 · British

Fisher is widely regarded as the father of modern experimental design and causal inference methodology. Working at Rothamsted Experimental Station in the 1920s, he developed the principles of randomization, replication, and blocking that underpin all valid experiments. His 1935 book 'The Design of Experiments' introduced the randomized controlled trial concept and the logic connecting experimental design to causal conclusions. He also developed the F-test and analysis of variance (ANOVA) to compare treatment effects. Fisher once settled a dispute about whether a woman could taste if milk was added to tea before or after — by designing a formal randomized experiment on the spot. This 'Lady Tasting Tea' story is now a classic introduction to experimental design in statistics courses worldwide.

  • Confounding Variables — Confounders are the main threat to causal inference in observational studies; randomization in experiments eliminates their effect, making this concept central to understanding why experiments outperform observations for causal claims.
  • Randomization and Sampling Methods — Random assignment is the cornerstone mechanism that makes causal inference valid in experiments; understanding sampling techniques clarifies how randomization balances both known and unknown confounders.
  • Hypothesis Testing and P-values — After computing the estimated treatment effect, hypothesis testing determines whether the observed difference is statistically significant or could plausibly be due to chance, completing the causal inference process.
  • Observational Studies vs. Experiments — The distinction between observational studies and experiments is fundamental: only properly designed experiments support causal claims, while observational studies reveal associations that may or may not reflect causation.