Accuracy, Certainty and Surprise - A Prediction Market on the Outcome of the 2002 FIFA World Cup

In this chapter, we present our empirical investigation of the forecasting accuracy of a prediction market experiment drawn on the outcome of the World Cup 2002. We analyse the predictive accuracy of 64 markets and compare to bookmakers’ quotes and chance as benchmarks. We revisit the evaluation of Schmidt and Werwatz (Chapter 16) and compare our results directly to their findings. In addition, we propose a new method for testing predictive accuracy by means of a non-parametric test for the similarity of probability distributions and we evaluate the incorporation of information in market prices by comparing pre-match and half-time price data. We find a reversed favourite-longshot bias when analysing market prices before the start of the match and this bias does not disappear with the inflow of new information until half-time. Unlike the market based predictions bookmakers appear to be perfectly calibrated. Since there were substantial deviations in outcome between the 2000 European Championship and our data, we offer possible explanations for the much worse performance of the 2002 World Cup prediction market. Consistent with Schmidt and Werwatz (Chapter 16) prediction markets do assign relatively higher probabilities to the favourite when compared to the odds-setters. Together with a long streak of surprising outcomes this fact appears most likely to be responsible for the predictive inaccuracy.