Discrete Random Variables and Distributions

One-line: A discrete random variable maps outcomes of a random experiment to isolated numerical values, and its distribution assigns a probability to each value via a probability mass function (PMF). Key result in words: The behaviour of any discrete random variable is completely determined by its PMF, which must be non-negative and sum to one; from it we compute every probability, expectation, variance, and moment. Where it applies: Counting problems, success/failure trials, arrival counts, queueing, genetics, reliability, machine learning classification, and any setting with countably many distinguishable outcomes. Who/when: Foundations laid by Pascal & Fermat (1654), Jacob Bernoulli (1713), Siméon Poisson (1837), with the modern measure-theoretic framework by Kolmogorov (1933).

A discrete random variable $X$ is a function from a sample space $Ω$ to a countable subset of the real numbers, such that the preimage of every value is an event with a well-defined probability. Its distribution is encoded by the probability mass function $p_{X} (x) = P (X = x)$ , which records exactly how the total probability of $1$ is distributed across the possible values. This topic matters because an enormous fraction of applied probability — from counting defective items to modelling photon arrivals — concerns quantities that take only separated values. A surprising fact: although the values are isolated, the cumulative distribution function of a discrete variable is a right-continuous step function that still carries all the information of the PMF, and many "continuous-looking" approximations (e.g. the normal approximation to the binomial) emerge from discrete distributions in the limit.

The Core Insight

A discrete random variable is fully captured by a list of (value, probability) pairs. Every question you can ask — "what is the chance $X$ exceeds 3?", "what is its average?", "how spread out is it?" — is just a weighted sum over that list. The PMF is the master object; everything else (CDF, mean, variance, moment generating function) is derived from it by summation.

Formal Statement

Random Variable and Discreteness

Let $(Ω, F, P)$ be a probability space. A random variable is a measurable function $X : Ω \to R$ , meaning that for every real $x$ , the set ${ω \in Ω : X (ω) \leq x}$ belongs to $F$ .

X

is discrete if its image (range) is a finite or countably infinite set

X = {x_{1}, x_{2}, x_{3}, \dots} \subset R

$Ω$ — sample space (set of all outcomes)
$F$ — $σ$ -algebra of events
$P$ — probability measure, $P : F \to [0, 1]$
$X (ω)$ — the numerical value assigned to outcome $ω$
$X$ — the (countable) support of $X$

Probability Mass Function

The probability mass function (PMF) of a discrete random variable $X$ is

p_{X} (x) = P (X = x) = P ({ω : X (ω) = x}) .

⟳this is just the probability measure

P

evaluated on the event

{X = x}

, the preimage

X^{- 1} ({x})

A valid PMF satisfies two axioms:

p_{X} (x) \geq 0 \forall x, x \in X \sum p_{X} (x) = 1.

⟳non-negativity comes from

P \geq 0

; the sum is

P (Ω) = 1

because the events

{X = x_{i}}

partition

Ω

(countable additivity).

For any subset $A \subseteq R$ ,

P (X \in A) = x \in A \cap X \sum p_{X} (x) .

⟳

{X \in A}

is the disjoint union over

x \in A \cap X

{X = x}

; apply countable additivity.

Cumulative Distribution Function

The cumulative distribution function (CDF) is

F_{X} (x) = P (X \leq x) = x_{i} \leq x \sum p_{X} (x_{i}) .

⟳

{X \leq x} = ⋃_{x_{i} \leq x} {X = x_{i}}

, a disjoint union; sum the masses.

For a discrete variable, $F_{X}$ is a right-continuous, non-decreasing step function with jumps of size $p_{X} (x_{i})$ at each $x_{i}$ . The PMF is recovered by

p_{X} (x_{i}) = F_{X} (x_{i}) - F_{X} (x_{i}^{-}) = F_{X} (x_{i}) - t ↑ x_{i} lim F_{X} (t) .

⟳the jump at

x_{i}

equals

P (X \leq x_{i}) - P (X < x_{i}) = P (X = x_{i})

Expectation, Variance, Moments

The expected value (mean) of $X$ , when the sum converges absolutely, is

E [X] = μ = x \in X \sum x p_{X} (x) .

⟳a probability-weighted average; it is the Lebesgue integral

\int X d P

reduced to a sum for the discrete case.

For a function $g$ , the Law of the Unconscious Statistician gives

E [g (X)] = x \in X \sum g (x) p_{X} (x) .

⟳group outcomes by the value of

X

; one need not first find the PMF of

g (X)

The variance measures spread:

Var (X) = σ^{2} = E [(X - μ)^{2}] = E [X^{2}] - μ^{2} .

⟳expand

(X - μ)^{2} = X^{2} - 2 μ X + μ^{2}

and use linearity of

E

, with

E [μ] = μ

The probability generating function (PGF), for $N$ -valued $X$ , is

G_{X} (s) = E [s^{X}] = k = 0 \sum \infty s^{k} p_{X} (k), ∣ s ∣ \leq 1.

⟳apply [7] with

g (x) = s^{x}

; derivatives at

s = 1

recover factorial moments.

Catalogue of Standard Discrete Distributions

The principal named families, with PMF, mean, and variance:

Bernoulli( $p$ ):

p_{X} (1) = p, p_{X} (0) = 1 - p

;

μ = p, σ^{2} = p (1 - p)

. Binomial( $n, p$ ):

p_{X} (k) = (k n) p^{k} (1 - p)^{n - k}, k = 0, \dots, n; μ = n p, σ^{2} = n p (1 - p) .

⟳

X = \sum_{i = 1}^{n} B_{i}

of i.i.d. Bernoullis; the binomial coefficient counts arrangements of

k

successes.

Geometric( $p$ ) (trials until first success,

k \geq 1

p_{X} (k) = (1 - p)^{k - 1} p; μ = \frac{1}{p}, σ^{2} = \frac{1 - p}{p ^{2}} .

⟳

k - 1

failures then a success; geometric series gives normalisation and mean.

Poisson( $λ$ ):

p_{X} (k) = \frac{λ ^{k} e ^{- λ}}{k !}, k = 0, 1, 2, \dots; μ = λ, σ^{2} = λ .

⟳limit of Binomial(

n, λ / n

) as

n \to \infty

; normalisation uses the series

\sum λ^{k} / k! = e^{λ}

Why It Holds / Derivation

Linearity of Expectation and the Binomial Mean

We prove $E [a X + bY] = a E [X] + b E [Y]$ and use it for the binomial.

Step 1. Let

X, Y

be discrete with joint PMF

p_{X, Y} (x, y)

. By the LOTUS [7] applied to

g (x, y) = a x + b y

E [a X + bY] = x \sum y \sum (a x + b y) p_{X, Y} (x, y) .

Step 2. Split the sum and factor constants:

= a x \sum x y \sum p_{X, Y} (x, y) + b y \sum y x \sum p_{X, Y} (x, y) .

Step 3. The inner sums are the marginals:

\sum_{y} p_{X, Y} (x, y) = p_{X} (x)

and

\sum_{x} p_{X, Y} (x, y) = p_{Y} (y)

. Hence

= a x \sum x p_{X} (x) + b y \sum y p_{Y} (y) = a E [X] + b E [Y] .

No independence was assumed. Step 4 (Binomial mean). Write

X = \sum_{i = 1}^{n} B_{i}

where

B_{i} \sim Bernoulli (p)

. Since

E [B_{i}] = p

, linearity gives

E [X] = \sum_{i = 1}^{n} p = n p

■

Poisson as a Limit of the Binomial

Step 1. Take Binomial(

n, p

) with

p = λ / n

held so that

n p = λ

is fixed, and let

n \to \infty

p_{X} (k) = (k n) (\frac{λ}{n})^{k} (1 - \frac{λ}{n})^{n - k} .

Step 2. Expand

(k n) = \frac{n ( n - 1 ) \dots ( n - k + 1 )}{k !}

and group:

= \frac{λ ^{k}}{k !} \cdot \to 1 \frac{n ( n - 1 ) \dots ( n - k + 1 )}{n ^{k}} \cdot (1 - \frac{λ}{n})^{n} (1 - \frac{λ}{n})^{- k} .

Step 3. As

n \to \infty

: the falling-factorial ratio

\to 1

, the last factor

\to 1

, and

(1 - λ / n)^{n} \to e^{- λ}

. Step 4. Therefore

p_{X} (k) \to \frac{λ ^{k} e ^{- λ}}{k !}

, the Poisson PMF [12].

■

Siméon Denis Poisson | 1781–1840

A French mathematician and physicist, Poisson introduced the distribution now bearing his name in his 1837 treatise Recherches sur la probabilité des jugements en matière criminelle et en matière civile, where he studied the number of wrongful convictions. He framed it as the limit of rare events in many independent trials — exactly the "law of small numbers." Poisson also made deep contributions to electromagnetism, elasticity, and celestial mechanics; the distribution was something of an aside in his work, yet it became one of the most-used models in all of science, governing radioactive decay counts, telephone-call arrivals, and biological mutation rates.

Building Intuition

Picture the total probability as one kilogram of sand. A discrete distribution places the sand in distinct piles at separated points on the number line; a continuous distribution would spread it smoothly. The PMF tells you the weight of each pile. The CDF is what you get by walking left-to-right and accumulating sand — it stays flat between piles and jumps up by the pile's weight whenever you step on one, hence the staircase shape.

The mean is the balance point of all the piles (a centre of mass). The variance is the average squared distance of sand from that balance point — small if the piles huddle near the mean, large if they sprawl. The Bernoulli is the atom: a single pile-pair at 0 and 1. The Binomial stacks $n$ such atoms; the Poisson is what the binomial becomes when each pile is tiny but there are very many — the "rare-events" regime.

Worked Examples

Quality Control — Binomial

Given: A factory produces widgets, each defective independently with probability

p = 0.05

. A box holds

n = 20

widgets. Let

X

= number of defectives. Question: Find

P (X = 2)

E [X]

Var (X)

, and

P (X \geq 1)

. Steps:

$X \sim Binomial (20, 0.05)$ . Using [10]:

P (X = 2) = (2 20) (0.05)^{2} (0.95)^{18} = 190 \times 0.0025 \times 0.3972 = 0.1887.

Mean: $E [X] = n p = 20 (0.05) = 1.0$ defectives.
Variance: $Var (X) = n p (1 - p) = 20 (0.05) (0.95) = 0.95$ , so $σ \approx 0.975$ .
At least one defective: $P (X \geq 1) = 1 - P (X = 0) = 1 - (0.95)^{20} = 1 - 0.3585 = 0.6415.$

Answer:

P (X = 2) \approx 0.189, μ = 1.0, σ^{2} = 0.95, P (X \geq 1) \approx 0.642

Server Requests — Poisson

Given: A web server receives requests at an average rate of

λ = 3

per second. Let

X

= number of requests in a one-second window, modelled as

X \sim Poisson (3)

. Question: Find

P (X = 0)

P (X \leq 2)

, and verify the variance. Steps:

From [12]: $P (X = 0) = \frac{3 ^{0} e ^{- 3}}{0 !} = e^{- 3} = 0.0498.$
$P (X = 1) = 3 e^{- 3} = 0.1494$ ; $P (X = 2) = \frac{9}{2} e^{- 3} = 0.2240.$
$P (X \leq 2) = e^{- 3} (1 + 3 + 4.5) = e^{- 3} (8.5) = 0.4232.$
Variance equals the mean: $Var (X) = λ = 3$ , so $σ = 3 \approx 1.732$ requests.

Answer:

P (X = 0) \approx 0.0498, P (X \leq 2) \approx 0.423, σ^{2} = 3 requests^{2}

PMF (bars) and CDF (staircase) of a Binomial(10, 0.3)

Common Misconceptions

PMF values vs. probabilities; CDF endpoints

A PMF value is itself a probability, $p_{X} (x) = P (X = x)$ , so it must lie in $[0, 1]$ . This differs from a continuous density, where the value $f (x)$ can exceed 1 and $P (X = x) = 0$ .
For discrete variables, strict vs. non-strict inequalities matter: $P (X < 3) \neq = P (X \leq 3)$ in general — they differ by $p_{X} (3)$ . Be especially careful when using $F_{X}$ .
The CDF is right-continuous with the value including the jump: $F_{X} (x) = P (X \leq x)$ takes the top of the step, not the bottom.
$E [g (X)] \neq = g (E [X])$ in general (Jensen's inequality); e.g. $E [X^{2}] \neq = (E [X])^{2}$ unless $Var (X) = 0$ .

A mean that does not exist

Consider $X$ with PMF $p_{X} (k) = \frac{6}{π ^{2} k ^{2}}$ for $k = 1, 2, 3, \dots$ (valid since $\sum 1/ k^{2} = π^{2} /6$ ). The expectation

E [X] = k = 1 \sum \infty k \cdot \frac{6}{π ^{2} k ^{2}} = \frac{6}{π ^{2}} k = 1 \sum \infty \frac{1}{k} = \infty

diverges (harmonic series). So a perfectly legitimate discrete distribution can have no finite mean. The St. Petersburg game is the classic gambling instance: a finite, well-defined game with infinite expected payoff. Moral: "every distribution has a mean" is false; existence requires absolute convergence of [6].

Connections

Discrete random variables sit inside the broader theory of Probability Measures and Sigma-Algebras, which make the word "measurable" in the definition precise. They contrast with Continuous Random Variables and Densities, where sums become integrals and PMFs become PDFs; the two are unified by the Lebesgue–Stieltjes Integral. Sums of independent discrete variables are handled by Convolution of Distributions and elegantly by the probability generating function [9]. The large- $n$ behaviour of binomial sums leads into the Central Limit Theorem (normal approximation) and the Law of Large Numbers. Conditional structure connects to Joint and Conditional Distributions and Bayes' Theorem.

Real-World Applications

Quality control & reliability — Binomial models the count of defective items in a batch; acceptance sampling plans set thresholds on $P (X \geq k)$ .
Telecommunications & queueing — Poisson governs call/packet arrivals (the basis of the Erlang formulas used to size telephone exchanges and data networks).
Epidemiology & genetics — Binomial and Poisson model rare mutation counts and disease incidence (the Luria–Delbrück experiment used the Poisson to demonstrate spontaneous bacterial mutation).
Insurance & actuarial science — Poisson (or negative binomial) models the number of claims per policy per year, feeding aggregate-loss calculations.
Machine learning — The categorical/multinomial distribution underlies softmax classifiers, Naive Bayes text models (word counts), and topic models (LDA).
Particle physics & astronomy — Geiger-counter clicks and photon counts in a CCD pixel are Poisson, defining the "shot-noise" floor of detectors.
Web analytics & A/B testing — Conversion counts among $n$ visitors are Binomial; hypothesis tests compare two proportions.

Historical Development

1654 — Pascal and Fermat correspond on the problem of points, implicitly using binomial-coefficient counting; birth of probability.
1657 — Christiaan Huygens publishes De Ratiociniis in Ludo Aleae, the first printed probability text, introducing expectation.
1713 — Jacob Bernoulli's posthumous Ars Conjectandi states the (weak) law of large numbers and systematises the binomial distribution.
1738 — Abraham de Moivre derives the normal approximation to the binomial, a precursor to the CLT.
1837 — Siméon Poisson introduces the Poisson distribution as a limit of rare binomial events.
1860s–1870s — Maxwell and Boltzmann use discrete counting in statistical mechanics; later the Bose–Einstein and Fermi–Dirac statistics.
1898 — Ladislaus Bortkiewicz publishes Das Gesetz der kleinen Zahlen, fitting the Poisson to deaths by horse-kick in the Prussian army — a famous empirical validation.
1933 — Andrey Kolmogorov's Grundbegriffe der Wahrscheinlichkeitsrechnung gives the axiomatic, measure-theoretic foundation, making "random variable" rigorous.
1940s–1950s — Feller's influential textbook An Introduction to Probability Theory and Its Applications canonises the modern treatment of discrete distributions.

Subtleties & Limits

Boundaries, infinities, and modelling fidelity

Several genuine subtleties persist beyond the clean undergraduate picture. (1) Existence of moments: as the counterexample shows, means and variances may diverge for heavy-tailed discrete laws (e.g. Zipf/zeta distributions ubiquitous in linguistics and network science); choosing summary statistics then requires care. (2) Mixed and singular distributions: real data may combine a discrete atom (e.g. exactly-zero insurance claims) with a continuous part, requiring the Lebesgue–Stieltjes Integral for a unified treatment. (3) Overdispersion: empirical count data often have variance exceeding the mean, violating the Poisson's $σ^{2} = μ$ identity and motivating the negative binomial — diagnosing the right model from finite data is an open practical challenge. (4) Infinite-support inference: estimating $λ$ or unseen species counts (the Good–Turing problem) from a finite sample remains an active area in statistics and information theory. (5) High-dimensional categorical data: modelling joint distributions over many discrete variables suffers the curse of dimensionality, driving research in graphical models and contrastive estimation.