Categorical Data

The Core Insight

You cannot compute a meaningful mean of categories like {dog, cat, fish}, but you can count how often each occurs and compare those counts to what pure chance would predict. Categorical data is information about membership, and its analysis is fundamentally about counts, proportions, and association rather than magnitudes.

Formal Statement

Categorical Variable

A categorical variable is a variable whose possible values form a finite (or countable) set of categories (also called levels) with no inherent arithmetic meaning. Two sub-types exist:

• Nominal: categories have no natural order (e.g., blood type ∈ {A, B, AB, O}).
• Ordinal: categories have a meaningful order but unequal/undefined spacing (e.g., {poor, fair, good, excellent}).

If a variable $X$ takes value in a category set $\mathcal{C}=\{c_1,c_2,\dots ,c_k\}$, then for $n$ observations the frequency (count) of category $c_i$ is

The relative frequency (sample proportion) of category $c_i$ is

When two categorical variables $X$ (with $r$ rows of categories) and $Y$ (with $c$ columns) are recorded together, the data forms a contingency table with cell counts $O_{ij}$:

Expected Counts Under Independence

If $X$ and $Y$ are statistically independent, the expected count in cell $(i,j)$ is

The chi-square statistic measuring departure from independence is

which, for large samples under independence, follows approximately a chi-square distribution with $(r-1)(c-1)$ degrees of freedom.

Why It Holds / Derivation

Why the Expected-Count Formula Works

We derive equation [4] for the expected cell count under independence.

1. Estimate marginal probabilities. The proportion of observations in row category $i$ is estimated by ${\hat{p}}_{X=i}=R_i/n$. Likewise the proportion in column category $j$ is ${\hat{p}}_{Y=j}=C_j/n$.

1. Apply independence. If $X$ and $Y$ are independent, the probability of landing in cell $(i,j)$ is the product:

1. Convert probability to expected count. With $n$ total observations, the expected number falling in cell $(i,j)$ is $n$ times the cell probability:

1. Conclusion. This is exactly equation [4]. Deviations of observed $O_{ij}$ from these $E_{ij}$ signal that the variables are not independent — quantified by equation [5]. $■$

Karl Pearson | 1857–1936

The English mathematician Karl Pearson founded much of modern mathematical statistics and, in 1900, introduced the chi-square goodness-of-fit test — the first rigorous tool for analyzing categorical counts. Working at University College London, Pearson sought to test whether observed frequency data (such as outcomes of dice rolls or biological classifications) matched a theoretical model. His statistic, equation [5], turned vague impressions about "fit" into a precise probability. Pearson also coined the term "contingency table" (in 1904) and developed measures of association for categorical variables. Though some of his views and disputes (notably with R.A. Fisher and his promotion of eugenics) are now heavily criticized, his statistical machinery remains foundational.

Building Intuition

Think of categorical data as sorting items into labeled buckets. You cannot ask "what is the average bucket?" — the question is nonsense. Instead you ask: how full is each bucket? (frequencies), what fraction is in each? (proportions), and does filling one bucket of variable A tell you anything about the buckets of variable B? (association).

The chi-square test is like comparing the actual contents of a two-way grid of buckets against the contents you'd predict if the two labelings were unrelated. If a survey shows 50% of people are coffee-drinkers and 50% are tea-drinkers, and 50% are morning people, then under independence you'd expect about 25% to be coffee-drinking morning people. If you actually observe 40%, the mismatch between observed and expected hints that coffee preference and morning-ness are linked.

Bar chart of nominal data vs. a two-way contingency structure

Worked Examples

Proportions and the Mode of a Nominal Variable

1. Apply equation [2]: ${\hat{p}}_{\text{Red}}=40/118=0.339$.
2. ${\hat{p}}_{\text{Blue}}=52/118=0.441$.
3. ${\hat{p}}_{\text{Green}}=26/118=0.220$.
4. Check sum: $0.339+0.441+0.220=1.000$ ✓ (consistent with equation [2]).
5. The mode is the category with highest count: Blue (52). The mean is undefined — colors have no numeric value.

Chi-Square Test of Association

| | Sleep Yes | Sleep No | Row total | |---|---|---|---| | Exercise Yes | 60 | 30 | 90 | | Exercise No | 40 | 70 | 110 | | Col total | 100 | 100 | 200 |

1. Compute expected counts via equation [4]:

1. Apply equation [5] term by term:

1. Sum: ${\chi }^2=5.00+5.00+4.09+4.09=18.18$.
2. Degrees of freedom: $(r-1)(c-1)=(2-1)(2-1)=1$. The critical value at the 5% level for df = 1 is $3.84$.
3. Since $18.18\gg 3.84$, reject independence.

Common Misconceptions

Numbers That Are Actually Categorical

Postal/ZIP codes, jersey numbers, telephone numbers, and "Route 66" all look numerical but are categorical (nominal). The "average ZIP code" of a group of customers is nonsense — ZIP 90210 is not "twice" ZIP 45105. Conversely, "number of children" (0, 1, 2, 3, …) looks like it could be a category but is genuinely numerical (discrete) because averaging makes sense (a mean of 2.3 children is meaningful). The test is not whether digits appear, but whether arithmetic carries real meaning.

Connections

Categorical data sits at the foundation of descriptive and inferential statistics. It pairs with Numerical Data (its quantitative counterpart) to complete the typology of variables, and its summaries feed directly into Probability Distributions — specifically the Binomial Distribution (two categories) and Multinomial Distribution (many categories). The analysis tools connect to Hypothesis Testing through the Chi-Square Distribution, and the underlying logic of counting outcomes connects to basic Probability. In machine learning, categorical variables require One-Hot Encoding before use in numerical models, linking this topic to Data Preprocessing.

Real-World Applications

1. Medicine — clinical trials: Outcomes coded as {recovered, no change, worsened}; chi-square tests compare treatment vs. placebo groups.
2. Genetics — Mendelian ratios: Pearson's original chi-square test checked whether observed offspring phenotype counts (e.g., 9:3:3:1) match predicted Mendelian inheritance.
3. Market research — survey analysis: Customer responses ({satisfied, neutral, dissatisfied}) cross-tabulated against demographics to find target segments.
4. Epidemiology — risk factors: Contingency tables of {exposed/unexposed} vs. {diseased/healthy} yield odds ratios and chi-square tests (e.g., smoking vs. lung cancer).
5. Elections & polling: Vote choice and party affiliation are categorical; pollsters report proportions with margins of error.
6. Machine learning — classification: Spam/not-spam, image labels (cat/dog/bird), and credit {approve/deny} are categorical target variables predicted by classifiers.
7. Quality control — defect types: Manufacturing logs classify defects by category (scratch, dent, misprint) to find dominant failure modes via Pareto charts.

Historical Development

• 1662 — John Graunt analyzes London's "Bills of Mortality," tabulating deaths by cause (a categorical classification), founding demographic statistics.
• 1786 — William Playfair invents the bar chart, the canonical display for categorical frequencies.
• 1880s — Francis Galton develops correlation and contingency ideas while studying heredity and classification.
• 1900 — Karl Pearson publishes the chi-square goodness-of-fit test, the first rigorous test for categorical count data.
• 1904 — Pearson introduces the term "contingency table" and measures of contingency (association).
• 1900s–1920s — George Udny Yule develops the Yule's Q and association coefficients for 2×2 tables.
• 1922 — R.A. Fisher clarifies the degrees of freedom for chi-square tests, correcting earlier errors.
• 1934 — Fisher's exact test provides a precise alternative to chi-square for small samples.
• 1950s–1970s — Log-linear models (Birch, Bishop, Fienberg, Holland) generalize contingency-table analysis to many dimensions.

Subtleties & Limits

The chi-square approximation in equation [5] is asymptotic — it works well only for large samples. A common rule of thumb requires every expected count $E_{ij}\ge 5$; when cells are sparse, the chi-square distribution poorly approximates the true distribution of the statistic, and Fisher's exact test or permutation methods are needed instead. Deeper open questions include: how best to analyze high-dimensional categorical data (many variables, each with many levels), where contingency tables become enormous and mostly empty (the "curse of dimensionality"); how to handle ordinal data in a way that respects order without falsely assuming equal spacing; and how to define and measure causal (not merely associational) effects among categorical variables — an active frontier in modern causal inference.

See Also