Why trust us
The model was replayed on the elections of April 2019, November 2019 and 2023, each time without the tested election for uncertainty, biases, pollsters and geography. The final widening of the uncertainty was chosen knowing these elections. 38 party-by-election observations.
Strict record, with a widening chosen each time without the tested election: vote shares within the 90% interval for 35 results out of 38 and within the 50% interval for 19. Seats within the 90% interval for 37 and within the 50% interval for 24.
The four figures below use the retained widening, chosen knowing these elections: they are more favourable than the strict record.
37 of 38seat results within the 90% interval
13 of 13major parties within the 90% seat interval
36 of 38vote-share results within the 90% interval
0.068mean Brier score, against 0.167 for polls taken at face value
Mean absolute error of vote shares. April 2019: 0.79 points against 0.83 for the raw poll average. November 2019: 0.56 points against 0.58 for the raw poll average. 2023: 0.53 points against 0.66 for the raw poll average.
Our misses
- The 50% seat intervals are too wide: they contain the result in 24 cases out of 38.
- In 2023, the replayed model gave the right a 53 in 100 chance of reaching 176 seats. It fell short.
- In April 2019, with a widening chosen without that election, the 90% vote-share interval contained the result for only 75% of parties.
The test far from the vote
For each past election, we take each pollster's latest poll available at a given date before the vote, then check whether the result falls within the vote-share interval given by our uncertainty at that horizon. This test covers vote shares, not seats, and does not run through the whole model chain.
Elections covered: 2016, April 2019, November 2019 and 2023. This test is in sample: the election uncertainty and its widening were measured on these same elections. Its main purpose is to check that uncertainty grows enough far from the vote.
The limits
- Only three elections serve as tests: one miss weighs a third of the record.
- Cross-validation covers polls from the last fourteen days. Far from the vote, drift is only tested in sample. The distribution used for new lists and its increase are not tested.
- The Brier score rests on 6 events. That is too few for a ranking.
- The polls used in this run are all verified at the primary source. Historical polls are transcribed from secondary sources.
- No Spanish forecast by Le Millénaire has yet been published and then checked against the result.