Mean Square Ratio

TL;DR

• A comparative measure that quantifies an estimator’s precision relative to a chosen reference estimator using mean squared error (MSE).
• MSR is the MSE of the estimator divided by the MSE of the reference; values less than 1 indicate the estimator under consideration is superior.
• Requires a defined reference estimator or model for comparison.

Definition

The mean squared ratio (MSR) is defined as the ratio of the mean squared error of an estimator to the mean squared error of a reference estimator, used to determine whether the estimator under consideration is superior to the reference estimator:

$$\text{MSR} = \frac{\text{MSE}(\text{estimator})}{\text{MSE}(\text{reference estimator})}$$

Explanation

Mean squared error (MSE) measures how far estimated values are from true values by averaging the squares of the differences between estimates and the true values. The MSR compares two estimators by dividing the MSE of the estimator under consideration by the MSE of a chosen reference estimator. An MSR less than 1 indicates the estimator under consideration has a lower MSE than the reference and is therefore considered superior in terms of that precision metric.

Examples

Weighing with two scales

Suppose two methods estimate the weight of an object: a standard scale (reference) and a digital scale (estimator). If three estimates from the digital scale are 10, 9.8, and 9.9 pounds and the true weight is 10 pounds, the MSE is:

$$\text{MSE} = \frac{(10 - 10)^2 + (9.8 - 10)^2 + (9.9 - 10)^2}{3} = \frac{0^2 + (0.2)^2 + (0.1)^2}{3} = 0.03$$

The mean squared ratio using the standard scale as the reference is:

$$\text{MSR} = \frac{\text{MSE of digital scale}}{\text{MSE of standard scale}} = \frac{0.03}{\text{MSE of standard scale}}$$

If MSR < 1, the digital scale is superior to the standard scale in terms of MSE.

Comparing predictive models (hypothesis testing)

When comparing two statistical models for prediction (for example, two models predicting the stock market), one model can be used as the reference estimator and the MSR of the other model computed. An MSR less than 1 indicates the model under consideration is superior to the reference model in terms of MSE.

Use cases

• Weight estimation (comparing measurement methods)
• Hypothesis testing and model comparison (comparing predictive models)

Notes or pitfalls

• MSR requires a clearly defined reference estimator or model.
• Interpretation: MSR < 1 indicates the estimator under consideration has a lower MSE than the reference and is therefore superior by this metric.

Related terms

• Mean squared error (MSE)
• Estimator
• Reference estimator
• Hypothesis testing