Monte Carlo Maximum Likelihood (MCML)

TL;DR

• Uses random sampling to generate many candidate parameter sets and evaluates each with the likelihood function.
• Chooses the parameter set with the highest evaluated likelihood as the estimate.
• Particularly useful when analytic maximization is difficult or computationally intensive, and for complex or high-dimensional models.

Definition

Monte Carlo maximum likelihood (MCML) is a computational method used to estimate the maximum likelihood of a given set of parameters, given a set of observed data. This method uses random sampling to generate a large number of potential parameter sets, and then uses the likelihood function to evaluate the probability of each set given the observed data. The parameter set with the highest likelihood is chosen as the maximum likelihood estimate.

Explanation

MCML proceeds by randomly sampling a large number of candidate parameter sets. For each sampled set, the likelihood function is evaluated to determine the probability of the observed data under that parameterization. The sampled parameter set that yields the highest likelihood value is selected as the maximum likelihood estimate. MCML avoids the need to solve for the maximum likelihood analytically or via deterministic optimization, which can be advantageous for complex or high-dimensional problems. Because it relies on random sampling, MCML can produce estimates that are more robust than some deterministic optimization techniques.

Examples

Linear regression (height and weight)

Suppose a dataset contains measurements of height and weight for a sample of individuals. To estimate the parameters of a linear regression model predicting weight from height using MCML:

• Generate a large number of potential sets of regression coefficients (for example, slope and intercept) by random sampling.
• Evaluate the likelihood of the observed data for each coefficient set using the likelihood function.
• Select the coefficient set with the highest likelihood as the maximum likelihood estimate.

Coin toss (probability of heads)

Suppose a dataset consists of outcomes from a series of coin tosses. To estimate the probability of heads using MCML:

• Generate a large number of potential values for the probability of heads by random sampling.
• Evaluate the likelihood of the observed toss outcomes for each candidate probability using the likelihood function.
• Choose the value with the highest likelihood as the maximum likelihood estimate of the probability of heads.

Use cases

• Estimating parameters in statistical models where analytic maximization is difficult.
• Parameter estimation for complex or high-dimensional data.
• Applications in statistics and machine learning where sampling-based approaches are appropriate.

Notes or pitfalls

• MCML is useful when solving for the maximum likelihood explicitly is difficult or computationally intensive.
• Because MCML uses random sampling, it can provide a more robust estimate of the maximum likelihood than methods that rely on deterministic optimization algorithms.

Related terms

• Likelihood function
• Maximum likelihood estimate (MLE)