Method Of Moments

TL;DR

• Estimate distribution parameters by computing sample moments and matching them to population moments.
• Commonly uses the first two moments (mean and variance) but can use higher-order moments.
• Solve the resulting equations for the unknown population parameters.

Definition

The method of moments is a statistical technique that uses sample data to estimate the parameters of a population distribution.

Explanation

Moments are numerical summaries of a dataset that describe its shape, spread, and center. The first moment is the mean (center of the data) and the second moment is the variance (spread of the data). Higher-order moments can also be calculated, though this description focuses on the first two.

The method of moments proceeds by calculating sample moments from observed data, equating those sample moments to the corresponding population moments, and solving those equations for the population parameters.

For example, with sample moments of order 1 and 2:

$$\text{(sample first moment)} = \text{(population first moment)},\\ \text{(sample second moment)} = \text{(population second moment)}.$$

Examples

Normal distribution (sample of 10 observations)

Suppose we have a sample of 10 observations from a normal distribution with unknown mean and variance. To estimate the population mean and variance using the method of moments:

• Calculate the sample mean (the average of the observations).
• Calculate the sample variance (the sum of squared differences between each observation and the sample mean, divided by the number of observations minus one).

For a normal distribution, the population mean is set equal to the sample mean. The population variance is set equal to the sample variance multiplied by a correction factor of n/(n-1), where n is the number of observations in the sample. Thus the estimated population mean and variance are the sample mean and the sample variance multiplied by this correction factor:

$$\text{population variance} = \text{sample variance} \times \frac{n}{n-1}.$$

Bernoulli distribution (sample of 10 observations)

Suppose we have a sample of 10 observations from a Bernoulli distribution with unknown probability of success p. To estimate p using the method of moments:

• Calculate the sample mean (the proportion of observations that were successes).

For a Bernoulli distribution, the population mean equals the probability of success p. Thus the estimated value of p is the sample mean.

Use cases

• Making inferences about a population based on a sample (useful for researchers and analysts).

Notes or pitfalls

• Higher-order moments can be used, but the explanation above focuses on the first two moments (mean and variance).

Related terms

• Moments
• Mean
• Variance
• Sample moments
• Population parameters
• Normal distribution
• Bernoulli distribution
• Sample mean
• Sample variance