Logistic Regression for Binary Outcomes

Does a smoker eventually develop lung cancer, yes or no? Does a seed germinate, or not? Does a patient survive surgery, or not? Biology is full of outcomes that come in exactly two flavors. A central question of quantitative biology is how such a yes/no fate depends on predictors. These predictors may be continuous or categorical — dose, age, temperature, genotype. Ordinary Multiple Regression: An Introduction assumes the response is continuous. It also assumes the response is roughly normally distributed around a straight-line (or plane) prediction. A binary outcome coded 0/1 violates both assumptions. Its variance depends mathematically on its mean. A straight line will cheerfully predict probabilities below 0 or above 1. This is biological nonsense. Logistic regression repairs this problem. It does not model the outcome directly. Instead, it models a transformed version of the probability of the outcome. It uses a link function that keeps predictions bounded between 0 and 1. This holds no matter how extreme the predictor. The payoff is enormous. Logistic regression underlies epidemiological risk models (does exposure X predict disease Y?). It also underlies toxicology dose-response curves and conservation models of habitat occupancy. Clinical prediction scores used at the bedside rely on it every day. It also introduces a way of thinking — modeling odds and their ratios. This way of thinking recurs throughout survival analysis, case-control studies, and generalized linear models generally. This article builds the logistic model from first principles. It explains why the logit link is the natural choice. It shows how maximum likelihood estimates its coefficients (there is no closed-form formula, unlike ordinary regression). It explains how to interpret coefficients as odds ratios. Finally, it shows how the classic evidence established this machinery as the standard tool. This evidence ranges from smoking-and-cancer studies to modern clinical trials. Together, it made logistic regression standard for binary biological data.

Key takeaways

Logistic regression models the probability of a binary (0/1) outcome as a function of predictors, keeping predicted probabilities strictly between 0 and 1.
It uses the logit link: log-odds = ln[p /(1−p)] is modeled as a linear function of predictors.
Coefficients (β) are estimated by maximum likelihood, not least squares; exponentiating a coefficient gives an odds ratio (OR).
An OR of 2 means the odds of the outcome double per one-unit increase in the predictor — it is not the same as a doubled probability.
Model fit is judged by likelihood-ratio tests, Wald tests, deviance, and discrimination measures like the ROC/AUC, not R².
The classic evidence base includes retrospective case-control smoking studies (Doll & Hill) and the prospective Framingham Heart Study, both analyzed with logistic-type models to quantify disease risk — laying groundwork that leads naturally into Survival Analysis and Kaplan-Meier: An Introduction for time-to-event data.

The core insight

Logistic regression predicts the log-odds of a yes/no outcome as a straight line in the predictors, so that back-transforming always yields a valid probability between 0 and 1, and each coefficient's exponential is an interpretable odds ratio.

The core idea

Suppose $Y$ is a binary response coded 1 for "event occurs" and 0 for "event does not occur" — infected/not infected, germinated/not germinated, died/survived. Let $p = \Pr(Y=1 \mid X)$ be the probability of the event given one or more predictors $X_1, X_2, \dots, X_k$ . Because $p$ is bounded in $(0,1)$ , we cannot model it directly as $p = \beta_0 + \beta_1 X_1$ — that linear predictor is unbounded and will inevitably produce impossible values of $p$ for extreme $X$ . The fix is to model a transformation of $p$ that is unbounded: the logit, defined as the natural log of the odds,

\text{logit}(p) = \ln\!\left(\frac{p}{1-p}\right).

As $p$ ranges over $(0,1)$ , the odds $p/(1-p)$ range over $(0,\infty)$ , and the log-odds range over the entire real line $(-\infty,\infty)$ — exactly what a linear predictor produces. The logistic regression model is therefore

\ln\!\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_k X_k.

Solving for $p$ gives the logistic function,

p = \frac{e^{\beta_0+\beta_1X_1+\cdots}}{1+e^{\beta_0+\beta_1X_1+\cdots}} = \frac{1}{1+e^{-(\beta_0+\beta_1X_1+\cdots)}},

an S-shaped ("sigmoid") curve that smoothly approaches 0 as the linear predictor goes to $-\infty$ and 1 as it goes to $+\infty$ , but never quite reaches either bound. This sigmoid shape is not an arbitrary curve-fitting trick — it falls out algebraically from insisting that log-odds be linear in the predictors. The coefficients have a clean interpretation once exponentiated. If $\beta_1$ is the coefficient on predictor $X_1$ , then $e^{\beta_1}$ is the odds ratio: the factor by which the odds of the event multiply for each one-unit increase in $X_1$ , holding other predictors constant. An odds ratio of 1 means no association; above 1 means the predictor raises the odds; below 1 means it lowers them. Because probability and odds are related non-linearly, an odds ratio is not the same as a relative risk (a ratio of probabilities), though the two converge numerically when the outcome is rare (the "rare disease approximation" commonly invoked in case-control epidemiology).

Mechanism — how the model is actually fit and used

Step 1 — Specify the model. Choose predictors (continuous, e.g. dose in mg/kg; or categorical, e.g. treatment vs. control) and write the linear predictor $\eta = \beta_0 + \beta_1X_1 + \cdots + \beta_kX_k$ . This $\eta$ is called the linear predictor on the logit scale.

Step 2 — Link to probability. The logit link connects $\eta$ to $p$ via $p = 1/(1+e^{-\eta})$ . Each individual observation $i$ , with its own predictor values, gets its own fitted probability $p_i$ , which is the probability that a Bernoulli trial for that individual yields $Y_i = 1$ .

Step 3 — Write the likelihood. Unlike ordinary regression, there is no error term added to $Y$ ; instead, $Y_i$ is modeled as a Bernoulli(p_i) random draw. The likelihood of the whole data set, given parameters $\beta_0,\dots,\beta_k$ , is

L(\boldsymbol\beta) = \prod_{i=1}^{n} p_i^{\,Y_i}(1-p_i)^{\,1-Y_i}.

This is the joint probability of observing exactly the pattern of 0s and 1s that occurred, given the model.

Step 4 — Maximize the likelihood numerically. Taking logs gives the log-likelihood

\ell(\boldsymbol\beta) = \sum_i \big[Y_i \ln p_i + (1-Y_i)\ln(1-p_i)\big].

Because $p_i$ depends non-linearly on $\boldsymbol\beta}$ through the logistic function, setting derivatives of $\ell$ to zero yields equations with no closed-form algebraic solution (unlike ordinary least squares, where a single matrix formula gives $\hat{\boldsymbol\beta}$ exactly). Instead, software uses an iterative algorithm — typically iteratively reweighted least squares (IRLS), a special case of Newton-Raphson — that starts from a guess, computes how the log-likelihood's slope and curvature look at that guess, updates the parameter estimates, and repeats until the estimates stop changing appreciably (convergence).

Step 5 — Quantify uncertainty and test hypotheses. At convergence, the curvature of the log-likelihood surface (the observed information matrix) yields standard errors for each $\hat\beta$ . A Wald test compares $\hat\beta/\text{SE}(\hat\beta)$ to a standard normal (or chi-square when squared) to test $H_0: \beta=0$ . A more robust alternative, the likelihood-ratio test, compares the log-likelihood of the full model to that of a reduced model lacking the predictor; twice the difference in log-likelihoods follows a chi-square distribution with degrees of freedom equal to the number of parameters dropped. This generalizes the logic of the F-test in Inference for Regression to models fit by maximum likelihood rather than least squares.

Step 6 — Assess fit and discrimination. Because residuals from a binary outcome do not behave like normal residuals, model fit is judged with deviance (−2 × log-likelihood, analogous to residual sum of squares), and predictive performance is often summarized by the ROC curve and its area (AUC), which measures how well the model's predicted probabilities separate events from non-events across all possible classification thresholds. An AUC of 0.5 indicates no better than chance; 1.0 indicates perfect discrimination.

Figure

The logistic (sigmoid) curve maps an unbounded linear predictor onto probability bounded in (0,1)

Figure

Odds ratio interpretation: exponentiated coefficient multiplies the odds, not the probability

The evidence — how logistic-type risk modeling was established

The intellectual roots of logistic regression trace to two threads: the mathematical logistic function, and the empirical demand — chiefly from mid-20th-century epidemiology — for a tool that could quantify disease risk from multiple predictors on a bounded probability scale.

Doll and Hill's smoking studies (1950, 1954, 1956). Richard Doll and Austin Bradford Hill conducted a landmark case-control study published in 1950, comparing 649 lung cancer patients in London hospitals against matched controls without lung cancer, examining smoking histories retrospectively. They found the proportion of heavy smokers was dramatically higher among cancer cases. Because the study was retrospective (cases and controls were sampled first, exposure ascertained after), the natural summary statistic was the odds ratio — computable directly from a 2×2 table without needing to know the underlying disease incidence in the population, a mathematical property that made odds ratios (and, later, logistic regression built around them) the natural currency of case-control epidemiology. Doll and Hill followed up with a landmark prospective cohort study of over 40,000 British doctors beginning in 1951, tracking actual smoking habits and subsequent mortality; by 1956 the dose-response relationship between cigarettes smoked per day and lung cancer mortality was unmistakable, and this cohort design allowed direct estimation of risk, later modeled with logistic and Poisson regression as those tools matured.

The Framingham Heart Study (initiated 1948). This prospective cohort study, following residents of Framingham, Massachusetts, across decades, generated the data that motivated much of the practical development of multivariable logistic regression in biomedicine during the 1960s-70s. Researchers needed to model the probability of a binary event — a heart attack within a defined follow-up period — as a joint function of continuous predictors (age, blood pressure, cholesterol) and binary ones (smoking status, sex). Ordinary linear regression on the 0/1 outcome (the "linear probability model") produced predicted probabilities outside [0,1] for some combinations of risk factors, an obviously unacceptable result for a clinical risk score. The adoption of the logistic model, with maximum-likelihood fitting made computationally feasible as electronic computing matured, resolved this and became the basis of the original Framingham Risk Score, still a template for cardiovascular risk calculators today.

What these studies collectively proved: 1. Binary disease outcomes can be linked quantitatively to continuous and categorical risk factors without the logical absurdity of probabilities outside [0,1] — validating the logit link as a mechanistic necessity rather than a convenience. 2. The odds ratio is a stable, computable summary of association in both retrospective (case-control) and prospective (cohort) designs, unifying the analysis of two very different epidemiological study architectures. 3. Multiple risk factors act (to good approximation, checkable and extendable) additively on the log-odds scale — the empirical justification for building multiple logistic regression rather than a series of one-variable comparisons, echoing the rationale of multiple regression itself.

Worked examples

Example 1 — Genetics-adjacent binary trait: penetrance and a logistic dose-response. Suppose a toxicologist studies a lethal mutation with incomplete, dose-dependent penetrance: carriers of a risk allele die from a chemical insult depending on the dose they receive, while non-carriers rarely die even at high dose. A logistic model is fit separately by genotype:

\text{logit}(p_{\text{death}}) = \beta_0 + \beta_1(\text{dose}) + \beta_2(\text{carrier}) + \beta_3(\text{dose}\times\text{carrier}).

Suppose fitted coefficients (dose in mg/kg) are:

\hat\beta_0 = -4.0,\quad \hat\beta_1 = 0.5,\quad \hat\beta_2 = 1.2,\quad \hat\beta_3 = 0.8.

For a non-carrier at dose = 6 mg/kg: $\eta = -4.0 + 0.5(6) = -1.0$ , so $p = 1/(1+e^{1.0}) = 0.269$ . For a carrier at the same dose: $\eta = -4.0 + 0.5(6) + 1.2(1) + 0.8(6)(1) = -4.0+3.0+1.2+4.8 = 5.0$ , so $p = 1/(1+e^{-5.0}) = 0.993$ . The carrier's odds of death at dose 6 are $0.993/0.007 \approx 142$ ; the non-carrier's odds are $0.269/0.731 \approx 0.368$ . The genotype-by-dose interaction ( $\hat\beta_3=0.8 > 0$ ) reveals that carriers are not just uniformly more susceptible — their susceptibility rises faster per unit dose, a synergy invisible to a model without the interaction term, and exactly the kind of asymmetric, gene-by-environment effect logistic regression is built to detect.

Example 2 — Clinical case-control scenario reasoned through by hand. A retrospective study of 200 patients undergoing a particular cardiac surgery examines whether diabetes status predicts 30-day post-operative mortality. Data:

	Died	Survived	Total
Diabetic	18	62	80
Non-diabetic	9	111	120

Odds of death given diabetic: $18/62 = 0.290$ . Odds of death given non-diabetic: $9/111 = 0.0811$ . Odds ratio $= 0.290/0.0811 = 3.58$ . Fitting a simple logistic regression with diabetes (1 = yes, 0 = no) as the sole predictor recovers this exactly: $\hat\beta_1 = \ln(3.58) = 1.276$ , and $e^{\hat\beta_1}=3.58$ is reported directly as the odds ratio, with a 95% Wald confidence interval computed from the standard error of $\hat\beta_1$ (here, roughly 1.50 to 8.53, meaning we can be reasonably confident the true odds ratio exceeds 1 — diabetes is associated with higher mortality odds). Note the risk difference framing: probability of death is $18/80 = 22.5\%$ among diabetics versus $9/120=7.5\%$ among non-diabetics — a relative risk of $22.5/7.5 = 3.0$ , close to but not identical to the odds ratio of 3.58. Because mortality here is not rare (22.5% is well above the "rare disease" threshold where OR ≈ RR), the odds ratio overstates the relative risk somewhat — a distinction clinicians must communicate carefully to patients who intuitively think in terms of probability, not odds.

Common misconceptions & traps

Misconception	Why it's wrong
"An odds ratio of 2 means the outcome is twice as likely (twice the probability)."	OR compares odds, not probabilities; the two only converge when the outcome is rare. At high baseline probability, OR can dramatically exceed the relative risk.
"Logistic regression coefficients can be interpreted like ordinary regression slopes, as constant changes in the outcome."	The effect of a one-unit predictor change on probability itself is not constant — it depends on where you are on the sigmoid curve; only the log-odds change is constant.
"You can fit logistic regression with ordinary least squares on 0/1 data and it's basically fine."	The "linear probability model" can predict impossible probabilities outside [0,1] and violates the assumption of constant-variance residuals, since binary variance is $p(1-p)$ , which changes with $p$ .
"A high pseudo-R² (like McFadden's) means the same thing as R² in linear regression."	Pseudo-R² values are not proportions of variance explained in the usual sense and are typically much lower than linear R² even for excellent binary models; they should be interpreted only in relative, not absolute, terms.
"Statistical significance of a predictor means it's clinically important."	A predictor can have a significant but tiny odds ratio (e.g., OR = 1.02) in a huge dataset; significance reflects sample size and precision, not effect magnitude.

The scientist behind it

The logistic function itself was introduced in 1838 by the Belgian mathematician Pierre-François Verhulst to model population growth that saturates at a carrying capacity, long before its statistical use for binary outcomes. Its adoption as a regression tool for probability data is usually credited to statisticians working in the mid-20th century, most notably David Cox, whose 1958 paper formalized logistic regression for binary response data and connected it to the broader theory of maximum likelihood inference that he and colleagues were developing. Cox's work sits alongside — and was deeply intertwined with — the epidemiological demand created by Doll, Hill, and the Framingham investigators for a principled way to model disease risk from multiple factors simultaneously. Sir David Cox (1924–2022) was a British statistician whose influence on quantitative biology and medicine is difficult to overstate: beyond logistic regression, he introduced the Cox proportional hazards model, the workhorse of survival analysis, directly foreshadowing the time-to-event methods that extend the binary-outcome logic of logistic regression into the domain of when an event happens, not just whether it happens — the subject taken up next in Survival Analysis and Kaplan-Meier: An Introduction.

Origins · 1 minSir David Cox and how this was discovered

Sir David Cox

1924–2022 · British statistician, Nuffield College Oxford and Imperial College London

Cox's 1958 formalization of logistic regression, and his 1972 proportional hazards paper, together supply the two dominant statistical frameworks for modern epidemiology and clinical biostatistics — one for whether an event occurs, one for when.

History

- 1838 — Verhulst introduces the logistic function for saturating population growth.

- 1950 — Doll and Hill publish the case-control study linking smoking to lung cancer, establishing the odds ratio as an epidemiological currency.

- 1951–1956 — The British Doctors Study (Doll & Hill) establishes prospective dose-response evidence for smoking and lung cancer mortality.

- 1958 — David Cox formalizes logistic regression as a maximum-likelihood model for binary response data.

- 1960s–70s — Framingham Heart Study investigators apply multivariable logistic regression to build the first quantitative cardiovascular risk scores.

Where this matters today

Logistic regression is the analytic backbone of clinical prediction tools: cardiovascular risk calculators descended from Framingham, sepsis and readmission-risk scores used in hospital electronic health records, and diagnostic classifiers that output a probability of malignancy from imaging or biomarker features. In epidemiology and public health, it remains the default model for case-control studies of disease exposure — from environmental toxins to vaccine safety surveillance — precisely because the odds ratio it estimates is valid regardless of how cases and controls were sampled. In genetics and genomics, logistic regression (often with many single-nucleotide-polymorphism predictors and correction for multiple testing) underlies genome-wide association studies (GWAS) that report the odds ratio of a disease per copy of a risk allele. In conservation biology and ecology, logistic regression models species presence/absence (occupancy) as a function of habitat variables, informing reserve design and endangered-species assessments. In agriculture and toxicology, it fits dose-response curves for pesticide efficacy or seed germination, generating the LD50/ED50 estimates central to regulatory risk assessment. And methodologically, logistic regression is the direct conceptual stepping-stone to the Cox proportional hazards model and Kaplan-Meier estimation, which handle the richer question of not just whether but when a binary event occurs amid censored observation times.

Ask the tutor: Can you walk through how to compute and interpret a 95% confidence interval for an odds ratio from logistic regression output?

Why do we model the logit of p instead of p itself?

Modeling $p$ directly with a linear equation allows predicted probabilities outside the valid range of 0 to 1 whenever predictors take extreme values — a mathematical impossibility for a true probability. The logit, $\ln[p/(1-p)]$ , maps the bounded interval $(0,1)$ onto the entire real line, so a linear equation on the logit scale can never generate an invalid back-transformed probability once the logistic function is applied. This is not a convenience but a structural necessity: any link function used for binary outcome modeling must have this property, and the logit is the canonical (and most interpretable, via odds ratios) choice among several that satisfy it, such as the probit.

What exactly is an odds ratio, and how is it different from relative risk?

Odds are $p/(1-p)$ , the ratio of the probability an event happens to the probability it doesn't. An odds ratio (OR) compares the odds under two conditions — e.g., exposed versus unexposed. Relative risk (RR) instead compares the raw probabilities $p_1/p_0$ . When the outcome is rare (say, under 10% probability in both groups), OR and RR are numerically close, which is why case-control studies of rare diseases can informally speak of OR as an approximate RR. When the outcome is common, OR moves further from RR and always further from 1 in the same direction, so OR should never be verbally translated as "times the risk" without checking baseline probabilities.

Can logistic regression handle more than one predictor, and how do we interpret coefficients then?

Yes — multiple logistic regression extends the single-predictor logit model to any number of predictors, exactly paralleling how Multiple Regression: An Introduction extends simple linear regression. Each coefficient $\beta_j$ , once exponentiated, gives the odds ratio for a one-unit increase in $X_j$ holding all other predictors constant — the same "adjusted effect" logic used in multiple linear regression to separate confounded variables. This is essential in observational biology and epidemiology, where exposures rarely occur in isolation (e.g., separating the effect of age from the effect of smoking on disease odds).

How do we know if a logistic regression model fits the data well?

There is no direct analog of R² because binary outcomes don't have a "variance explained" in the classical sense. Instead, biologists use several complementary tools: the deviance (a likelihood-based measure of lack of fit, decreasing as the model improves), likelihood-ratio tests comparing nested models, calibration plots (comparing predicted probabilities to observed event rates within bins), and discrimination measures like the ROC curve and its area (AUC), which quantify how well predicted probabilities separate true events from non-events across all possible decision thresholds. Good practice reports several of these together rather than a single summary number.

What is the difference between a Wald test and a likelihood-ratio test for a logistic regression coefficient?

Both test whether a coefficient differs from zero (equivalently, whether an odds ratio differs from 1), but they use different information. The Wald test uses the estimated coefficient divided by its standard error, squared to follow a chi-square distribution — quick to compute but can perform poorly with small samples or coefficients near boundary values. The likelihood-ratio test compares the log-likelihood of the model with the predictor to the log-likelihood of the model without it; this comparison is generally more reliable, especially in small-to-moderate samples, and is the preferred test when the two disagree.

Why does logistic regression foreshadow survival analysis?

Logistic regression answers "did the event happen by the end of follow-up, yes or no?" but ignores when it happened and cannot properly handle censoring — patients who leave a study, or whose follow-up ends, before the event occurs or before the study ends. Survival analysis, including Kaplan-Meier estimation and the Cox proportional hazards model, generalizes the binary-outcome logic to model the time until the event, explicitly accommodating censored observations. The same core habit of thought — modeling a rate or hazard on a transformed, unbounded scale and exponentiating coefficients into interpretable ratios — carries over directly, which is why logistic regression is typically taught immediately before survival methods.

Does a large odds ratio always mean the predictor causes the outcome?

No. Logistic regression, like any regression, models association, not automatically causation. A large, statistically robust odds ratio can still arise from confounding (an unmeasured variable driving both predictor and outcome), reverse causation, or selection bias in how cases and controls were recruited. Causal interpretation requires the same scrutiny applied to any observational analysis: consideration of study design, adjustment for plausible confounders, biological plausibility, consistency across independent studies (as with the convergence of case-control and cohort smoking studies), and ideally corroboration from randomized or quasi-experimental designs where ethically possible.