Skip to content

Explainers / MethodUpdated September 30, 2026

Model tests

We test whether the model recognises dangerous chances, whether player measures repeat, and whether the arithmetic adds up. Prediction is a separate question: useful evidence about completed games does not automatically support a next-season forecast.

Do the chance estimates mean what they say?

Imagine 100 attempts rated at a 10% chance of scoring. In this hypothetical example, we would expect about 10 goals, with some variation. Across many such groups, consistently getting far more or fewer goals would tell us the probabilities need work.

We also test whether the model puts the dangerous chances ahead of routine attempts. These are different checks. A model could correctly rank the chances while assigning probabilities that are too high to all of them.

That helps establish whether expected goals adds useful information beyond counting shots. It does not, by itself, establish which player deserves credit for creating those chances.

Keep each test’s purpose clear

Chance values were scored on 2025–26 shots the xG model had not learned from. That checks whether it ranks danger and assigns sensible probabilities. We also compare consecutive player-seasons to see how much each measure repeats.

Those are different checks. Repeatability is not proof that every assignment of individual credit is right, and a held-out test stops being independent once its results are used to choose a setting. For example, the xG training recency weighting used 2025–26 as one of the seasons for choosing among alternatives; that choice still needs a fresh future test.

Does the stated confidence hold up?

Getting the central estimate reasonably close and giving a dependable range are separate achievements.

Our historical review made this clear: ranges intended to contain 80% of outcomes covered only about 60% for stars, even one season ahead. A reasonable-looking result across all players had hidden a problem in a group that matters greatly to contract decisions.

Longer horizons brought further failures or too little evidence for a reliable judgment. That is why we do not publish supported outcome ranges or odds of earning back a whole contract.

Correct arithmetic comes first

We also check that goals, minutes and player identities line up, that the components add up, and that replacement is counted once. Missing data must stay missing.

Those checks protect the meaning of the numbers readers see. They do not prove that every bit of shared-ice credit belongs to the right player, or that a future prediction will come true.

The strongest next check is to fix predictions before the games and judge them afterward. Until that evidence arrives, we distinguish useful historical findings from a demonstrated future record—and keep the approaches that failed alongside the ones that worked.

Full method: How we test it