Sums of Random Variables and Convolution

Convolution of distributions is the working engine behind an enormous range of applied and theoretical results. In statistics, the distribution of a sample sum or sample mean of independent observations is a repeated convolution, and this underlies exact tests (e.g. exact distributions of sums of independent chi-squared or gamma variables) as well as the asymptotic machinery of the Central Limit Theorem. In signal processing and electrical engineering, convolution of probability densities is the probabilistic shadow of convolution of signals: adding independent noise sources to a signal literally convolves their density functions, and this is why engineers routinely move to the frequency domain via Characteristic Functions (Fourier transforms), where convolution becomes ordinary multiplication. In actuarial science and risk management, aggregate claim distributions are convolutions of individual claim-size distributions (compounded further over a random number of claims), and numerical convolution (or its transform-domain shortcut) is a standard computational tool. In statistical physics, sums of independent microscopic contributions (e.g. random walks, occupation numbers) are analyzed via convolution, connecting directly to the derivation of the Bivariate Normal Distribution as a stable limit of many small independent additive effects. In cryptography and coding theory, convolution of distributions over finite groups (a generalization of the real-line case) governs the analysis of linear feedback shift registers and differential cryptanalysis, where XOR-sums of independent bit-strings convolve mod-2 distributions. Finally, in pure probability, convolution stability (Poisson+Poisson=Poisson, Normal+Normal=Normal, Gamma+Gamma=Gamma with matching rate) is the first hint of the deeper theory of infinitely divisible distributions and stable laws.

Ask the tutor: Can you show me how to convolve two Gamma distributions with different shape parameters but the same rate, step by step?

Picture the joint distribution of $(X,Y)$ as mass or density spread over the plane. The event $\{X+Y = z\}$ (discrete) or $\{X+Y \in [z, z+dz]\}$ (continuous) is exactly the set of points lying on, or infinitesimally near, the diagonal line $x + y = z$. To find the total probability of landing on that diagonal, you sweep along it: for each possible value $x$ of $X$, the partner value must be $y = z - x$, and independence tells you the joint mass or density at that point is simply the product $f_X(x) f_Y(z-x)$. Summing or integrating that product over all $x$ is precisely the convolution formula — it is a weighted average of $f_Y$ shifted and flipped, with weights $f_X$. This "flip and slide" picture is why convolution is often visualized as sliding one density past another: as $z$ increases, the diagonal line $x+y=z$ moves outward, and the convolution integral tracks how much of the joint mass currently intersects it.

For discrete variables the same idea appears as a lattice sum: fix $n$, and the achievable pairs $(k, n-k)$ trace a discrete anti-diagonal.

Example 1 (discrete, asymmetric parameters): sum of two independent Poissons with different rates. Let $X \sim \mathrm{Poisson}(\lambda)$ and $Y \sim \mathrm{Poisson}(\mu)$ be independent, with $\lambda = 2$ and $\mu = 5$ (deliberately unequal, to avoid masking errors with symmetry). We compute the distribution of $Z = X+Y$ using the convolution formula and show $Z \sim \mathrm{Poisson}(\lambda+\mu) = \mathrm{Poisson}(7)$. By the Theorem, for $n = 0, 1, 2, \dots$,

(The sum truncates to $k=0,\dots,n$ since $p_Y(n-k)=0$ for $n-k<0$.) Factor out $e^{-(\lambda+\mu)}$ and multiply and divide by $n!$:

using the binomial theorem in the last step. With $\lambda+\mu = 7$, this says $Z \sim \mathrm{Poisson}(7)$ exactly. Let's verify a specific numeric value directly, without shortcuts, to make sure the formula is being used correctly: compute $p_Z(3) = P(X+Y=3)$ two ways. Direct convolution sum ($\lambda=2,\mu=5$):

Compute term by term ($e^{-2}e^{-5} = e^{-7}$ factored out at the end):

• $k=0$: $\frac{2^0}{0!}\cdot\frac{5^3}{3!} = 1 \cdot \frac{125}{6} = 20.8333$
• $k=1$: $\frac{2^1}{1!}\cdot\frac{5^2}{2!} = 2 \cdot 12.5 = 25$
• $k=2$: $\frac{2^2}{2!}\cdot\frac{5^1}{1!} = 2 \cdot 5 = 10$
• $k=3$: $\frac{2^3}{3!}\cdot\frac{5^0}{0!} = \frac{8}{6}\cdot 1 = 1.3333$

Sum: $20.8333 + 25 + 10 + 1.3333 = 57.1667$ (this equals $7^3/3! = 343/6 = 57.1\overline{6}$, confirming the binomial identity numerically). So

This is exactly $\mathrm{Poisson}(7)$'s formula at $n=3$: $e^{-7}7^3/3! \approx 0.0009119 \times 57.1667 \approx 0.05213$. Numerically, $e^{-7} \approx 0.00091188$, so $p_Z(3) \approx 0.00091188 \times 57.1667 \approx 0.052126$. This matches direct evaluation of the Poisson(7) pmf at 3, confirming the convolution computation is internally consistent.

Example 2 (continuous, asymmetric parameters): sum of an Exponential and a Uniform. Let $X \sim \mathrm{Exponential}(\lambda)$ with $\lambda = 2$ (density $f_X(x) = 2e^{-2x}$ for $x \ge 0$, else $0$), independent of $Y \sim \mathrm{Uniform}(0,1)$ (density $f_Y(y) = 1$ for $0 \le y \le 1$, else $0$). These are deliberately of different families with mismatched shapes. Find the density of $Z = X+Y$. By the Theorem,

The indicator restricts $x$ to $z - 1 \le x \le z$, intersected with $x \ge 0$. We split into cases based on $z$. Case $z < 0$: no overlap ($x\ge 0$ and $x \le z <0$ are incompatible), so $f_Z(z) = 0$. Case $0 \le z \le 1$: the constraint $x \ge \max(0, z-1) = 0$ (since $z - 1 \le 0$) and $x \le z$ gives integration range $[0, z]$:

Case $z > 1$: the constraint becomes $x \ge z-1 > 0$ and $x \le z$, so the range is $[z-1, z]$:

So altogether,

Sanity checks. Continuity at $z=1$: from the middle branch, $1-e^{-2} \approx 1 - 0.135335 = 0.864665$; from the top branch, $(e^2-1)e^{-2} = e^0 - e^{-2} = 1 - e^{-2} \approx 0.864665$. They agree, as they must since $f_Z$ is continuous (the convolution of a bounded density with an $L^1$ function is continuous). Total mass check:

First integral: $\left[z + \tfrac12 e^{-2z}\right]_0^1 = \left(1 + \tfrac12 e^{-2}\right) - \left(0+\tfrac12\right) = \tfrac12 + \tfrac12 e^{-2} \approx 0.5 + 0.0676675 = 0.567668$. Second integral: $(e^2-1)\int_1^\infty e^{-2z}dz = (e^2-1)\cdot \tfrac12 e^{-2} = \tfrac12(1 - e^{-2}) \approx 0.5(0.864665) = 0.432332$. Sum: $0.567668 + 0.432332 = 1.000000$. The total integrates to exactly $1$, confirming $f_Z$ is a genuine density.