Principles of Experimental Design

A confounding variable is one that is associated with both the explanatory variable and the response variable, making it impossible to tell which one caused the observed effect. Control means deliberately holding other variables constant or accounting for them in the design. The most common control strategy is using a control group — a group of subjects that does not receive the active treatment. In a drug trial, the control group receives a placebo (an inert pill that looks identical to the real drug). A double-blind design goes further: neither the subjects nor the researchers measuring outcomes know who received which treatment. This prevents both placebo effect (subjects improving because they believe they are being treated) and observer bias (researchers unconsciously scoring results in favour of the treatment they hope works).

Randomization is the process of using a random mechanism — like a random number generator, shuffled cards, or a coin flip — to assign subjects to treatment groups. Its power lies in this mathematical guarantee: over many experiments, randomization distributes both known and unknown lurking variables approximately equally across all groups. This means any difference in outcomes between groups is likely due to the treatment, not to pre-existing differences. Randomization also provides the mathematical foundation for statistical significance tests. Without it, we cannot legitimately apply probability calculations to our data. A simple randomization: label 40 volunteers 01–40, use a random number table to select 20 for the treatment group, and assign the remaining 20 to the control group. More sophisticated designs use stratified randomization — first dividing subjects into blocks (e.g., by age or sex) and then randomizing within each block — to ensure balance on important variables.

Replication means applying each treatment to multiple subjects (or experimental units). With only one subject per treatment, any observed difference might simply reflect that individual's unusual response. With many subjects, random variation averages out and the true treatment effect becomes visible. The required number of subjects depends on three factors: the desired significance level α (typically 0.05), the desired power 1 − β (the probability of detecting a true effect, typically 0.80), and the effect size d (how large a difference is scientifically meaningful). Larger samples are needed when effects are small or when we want high confidence. Replication is also distinguished from repeated measurements on the same subject, which does not guard against subject-specific effects the same way that independently replicating the whole experiment does.

Beyond the three core principles, experimental designs often incorporate blocking and blinding. Blocking groups similar experimental units together (a block) and then applies all treatments within each block. This reduces variability by controlling for known sources of variation. For instance, in an agricultural experiment testing three fertilisers, if field plots differ in soil quality, the researcher groups nearby plots into blocks of three and applies one fertiliser to each plot in a block. The matched pairs design is a special case of blocking with two treatments where each block contains exactly two experimental units (or one unit receives both treatments in random order). Blinding — keeping participants or evaluators unaware of treatment assignments — is a control mechanism that eliminates psychological biases from the data.

• n — Sample size per group (subjects): The number of experimental units assigned to each treatment group.
• z_{\alpha/2} — Significance critical value (dimensionless): The z-score corresponding to a two-tailed significance level α. For α = 0.05, z_{α/2} = 1.96.
• z_{\beta} — Power critical value (dimensionless): The z-score corresponding to the desired power 1 − β. For 80% power, z_β = 0.84.
• \sigma^2 — Population variance (units²): The variance of the response variable, estimated from prior studies or a pilot experiment.
• \delta — Minimum detectable difference (units): The smallest difference between group means that would be scientifically or practically meaningful.

Problem. A nutritionist wants to test whether a new protein supplement increases muscle mass more than a standard diet alone. She has 30 volunteers aged 16–18. She plans a completely randomized experiment with two groups: supplement (treatment) and no supplement (control). She will measure muscle mass gain in kilograms after 8 weeks. (a) Explain how she should use randomization to assign volunteers to groups. (b) Identify one potential confounding variable and explain how the design addresses it. (c) If prior research shows σ = 1.2 kg and the nutritionist wants to detect a difference of δ = 0.8 kg with α = 0.05 and 80% power, verify whether 30 total volunteers is sufficient.

Answer. 30 volunteers is insufficient; approximately 36 per group (72 total) are required.

• Sampling Methods — Experimental design depends on how subjects are selected from the population; understanding simple random, stratified, and cluster sampling clarifies who the conclusions apply to.
• Hypothesis Testing — The purpose of randomization in experimental design is to make hypothesis tests valid; significance levels and p-values quantify the evidence the experiment produces.
• Observational Studies and Bias — Contrasting observational studies with experiments highlights why confounding variables arise and why randomization is the crucial tool that separates correlation from causation.
• Analysis of Variance (ANOVA) — ANOVA is the statistical method used to analyse data from experiments with more than two treatment groups, directly applying the replication and blocking principles of experimental design.