The Method of Moments

In econometrics, the Generalized Method of Moments (GMM), developed by Lars Peter Hansen, extends this idea to estimate parameters from moment conditions implied by economic theory (e.g. rational expectations models), even when no full likelihood is specified. In actuarial science and insurance, method-of-moments fitting of Gamma, Lognormal, or Pareto claim-size distributions is routine because closed-form moment formulas make it fast to fit models to large policy datasets. In physics and engineering, moment matching is used to fit particle-size or noise distributions from measured moments (mean energy, variance) without needing a tractable likelihood. In numerical optimization for statistics, MoM estimates are the standard warm-start values fed into iterative algorithms (Newton–Raphson, EM) that compute Maximum Likelihood Estimation estimates, because a good starting point can be the difference between fast convergence and divergence. In signal processing, moment-based ("cumulant matching") methods estimate parameters of non-Gaussian signal models where likelihoods are analytically unavailable. Finally, in teaching, MoM is the natural first illustration of the general logic of Point Estimation and Estimators: observe a functional of the data, equate it to its theoretical counterpart, invert.

:::quiz[For i.i.d. Exponential($\lambda$) data with $E[X]=1/\lambda$, what is the method of moments estimator of $\lambda$?]

• $\hat\lambda = \bar X$
• $\hat\lambda = 1/\bar X$ ✓
• $\hat\lambda = n/\sum X_i^2$
• $\hat\lambda = \bar X^2$

Ask the tutor: How does the method of moments generalize into GMM when there are more moment conditions than parameters?

Think of the population as a machine that produces numbers according to some law $F_\theta$, and think of moments as fingerprints of that law: the mean tells you the central location, the second moment (combined with the mean) tells you the spread, the third tells you the asymmetry. As $n$ grows, the sample's own fingerprints — its sample mean, sample second moment, and so on — converge to the population's true fingerprints, by the Law of Large Numbers. The method of moments simply runs this correspondence backward: given the sample's fingerprints, ask which parameter value $\theta$ would have produced a population with exactly those fingerprints, and declare that value the estimate. Geometrically, $g$ maps the parameter space $\Theta$ into moment space $\mathbb{R}^k$. Each hypothetical parameter $\theta$ traces out a point $(\mu_1(\theta), \mu_2(\theta))$ in the plane (for $k=2$). The observed data gives us one specific point, the sample moment vector $(\hat\mu_1, \hat\mu_2)$, which typically does not lie exactly on the curve $g(\Theta)$ but lands very close to it once $n$ is large. We find the parameter whose theoretical point matches the data's observed point.

Example 1 — Gamma distribution (two parameters, asymmetric numbers). Let $X_1,\dots,X_n$ be i.i.d. Gamma$(\alpha,\beta)$ with density

where $\alpha>0$ is the shape and $\beta>0$ the scale. Standard moment formulas give

so the variance is $\operatorname{Var}(X) = E[X^2]-\mu_1^2 = \alpha\beta^2$. Suppose a sample of $n=8$ bulb lifetimes (in hundreds of hours) gives

Compute the sample mean:

Compute the sample second moment $\frac1n\sum x_i^2$:

Sample variance (dividing by $n$, matching the population second central moment):

Now solve $\alpha\beta = \hat\mu_1$ and $\alpha\beta^2 = \widehat{\operatorname{Var}}$. Dividing the second by the first:

So the method of moments gives $\hat\alpha \approx 7.84$, $\hat\beta \approx 0.606$ (hundreds of hours). Here the map $g(\alpha,\beta) = (\alpha\beta,\, \alpha(\alpha+1)\beta^2)$ is indeed injective on $(0,\infty)^2$: given $\mu_1 = \alpha\beta$ and $\operatorname{Var} = \alpha\beta^2$, the ratio $\operatorname{Var}/\mu_1 = \beta$ recovers $\beta$ uniquely, and then $\alpha = \mu_1/\beta$ is determined — confirming hypothesis (2) of the theorem is satisfiable in closed form here.

Example 2 — Uniform distribution on $(0,\theta)$ (a case where MoM and a natural alternative disagree strikingly). Let $X_1,\dots,X_n$ be i.i.d. Uniform$(0,\theta)$, $\theta>0$ unknown. Here $k=1$: we only need the first moment,

This $g(\theta) = \theta/2$ is trivially injective with inverse $g^{-1}(m) = 2m$. The method of moments estimator is

Take a concrete asymmetric sample of $n=6$ observations:

Then

so $\hat\theta^{\text{MoM}} = 2(3.0) = 6.0$. Notice something uncomfortable: the largest observed value is $x_{(6)} = 4.9 < 6.0$, so the MoM estimate is at least plausible (it must satisfy $\hat\theta \ge x_{(6)}$ to be consistent with the model, since no $x_i$ can exceed $\theta$ under Uniform$(0,\theta)$). But suppose instead the last observation had been $x_6 = 5.9$ (still $<6.0$, still consistent) — the mean would shift only slightly and MoM would still track reasonably. The real discomfort surfaces when comparing to the maximum likelihood estimator for this model, which is $\hat\theta^{\text{MLE}} = x_{(n)}$, the sample maximum — here $\hat\theta^{\text{MLE}} = 4.9$. The two estimators disagree substantially (6.0 vs. 4.9) on the very same data, illustrating concretely that MoM need not equal, or even approximate well, the MLE in small samples, foreshadowing the comparison of estimator efficiency and bias properties across methods. Both estimators are consistent as $n\to\infty$, but they can differ noticeably at finite $n$, and here MoM is clearly less efficient: it is a positively-weighted linear function of all the data whereas the sufficient statistic is just the maximum, so MoM discards information contained in the ordering of the sample.