ChiUng Song () (Science and Technology Policy Institute) Bryan L. Boulier () (Department of Economics The George Washington University) Herman O. Stekler () (Department of Economics The George Washington University)
Additional information is available for the following
registered author(s):
Previous research on defining and measuring consensus (agreement) among forecasters has been concerned with evaluation of forecasts of continuous variables. This previous work is not relevant when the forecasts involve binary decisions: up-down or win-lose. In this paper we use Cohen¡¯s kappa coefficient, a measure of inter-rater agreement involving binary choices, to evaluate forecasts of National Football League games. This statistic is applied to the forecasts of 74 experts and 31 statistical systems that predicted the outcomes of games during two NFL seasons. We conclude that the forecasters, particularly the systems, displayed significant levels of agreement and that levels of agreement in picking game winners were higher than in picking against the betting line. There is greater agreement among statistical systems in picking game winners or picking winners against the line as the season progresses, but no change in levels of agreement among experts. High levels of consensus among forecasters are associated with greater accuracy in picking game winners, but not in picking against the line.
Download Info
To download:
If you experience problems downloading a file, check if you have the
proper application to
view it first. Information about this may be contained
in the File-Format links below. In case of further problems read
the IDEAS help
page. Note that these files are not on the IDEAS
site. Please be patient as the files may be large.
Publisher Info
Paper provided by The George Washinton University, Department of Economics, Research Program on Forecasting in its series Working Papers with number
2008-6.