Skip to content
My Glossary Project

Brier Score

TL;DR

Section titled “TL;DR”
  • Quantifies how close predicted probabilities are to actual outcomes by squaring prediction errors and averaging them.
  • Lower values indicate more accurate probabilistic forecasts; 0 is perfect and 0.5 is cited for a completely random forecast.
  • Applicable to any probabilistic forecast (e.g., weather, markets, sports) and useful for comparing models or tracking performance over time.

Definition

Section titled “Definition”

The Brier Score (also called the Brier metric or Brier loss function) measures the accuracy of probabilistic predictions by taking the difference between the predicted probability and the observed outcome, squaring that difference, and averaging the squared differences over all predictions. It was developed by Glenn Brier in 1950.

Explanation

Section titled “Explanation”
  • For each probabilistic prediction, compute (predicted probability − observed outcome)^2.
  • Average these squared differences across all predictions to obtain the Brier Score.
  • The score indicates forecast accuracy: 0 corresponds to a perfect forecast; a completely random forecast is described as having a Brier Score of 0.5 in the source text.
  • The Brier Score can be used to compare forecasting models and to evaluate probabilistic forecasts over time.

Examples

Section titled “Examples”

Single prediction — rain forecast (rains)

Section titled “Single prediction — rain forecast (rains)”

If a forecast predicts a 75% chance of rain and it actually rains:

(0.75−1)2=0.0625(0.75 - 1)^2 = 0.0625

Single prediction — rain forecast (does not rain)

Section titled “Single prediction — rain forecast (does not rain)”

If a forecast predicts a 50% chance of rain and it does not rain:

(0.5−0)2=0.25(0.5 - 0)^2 = 0.25

Model comparison (same single outcome)

Section titled “Model comparison (same single outcome)”

Comparing two models predicting the same event where the event occurs:

  • Model A predicts 75%: Brier Score = 0.0625
  • Model B predicts 50%: Brier Score = 0.25 Model A is considered more accurate because it has the lower Brier Score.

Multiple predictions over time

Section titled “Multiple predictions over time”

Forecasts: 75% chance on Monday, 50% on Tuesday, 25% on Wednesday. Actual outcomes: rains on Monday and Tuesday, not on Wednesday. The Brier Score for this forecast is given in the source as:

(0.75−1)2+(0.5−1)2+(0.25−0)2=0.5625(0.75 - 1)^2 + (0.5 - 1)^2 + (0.25 - 0)^2 = 0.5625

Use cases

Section titled “Use cases”
  • Evaluating probabilistic forecasts in meteorology.
  • Assessing probabilistic predictions in finance (stock market) and sports betting.
  • Comparing the performance of different forecasting models.
  • Tracking forecast accuracy over time.

Notes or pitfalls

Section titled “Notes or pitfalls”
  • A perfect forecast has a Brier Score of 0.
  • The source describes a completely random forecast as having a Brier Score of 0.5.
  • The Brier Score is based on squared differences between predicted probabilities and observed binary outcomes.
Section titled “Related terms”
  • Brier metric
  • Brier loss function
  • Probabilistic forecasts
  • Glenn Brier (developer, 1950)