IDEAS home Printed from
MyIDEAS: Log in (now much improved!) to save this paper

Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging

  • Armantier, Olivier
  • Treich, Nicolas

Proper Scoring Rules (PSRs) are popular incentivized mechanisms to elicit an agent's beliefs. This paper combines theory and experiment to characterize how PSRs bias reported beliefs when (i) the PSR payments are increased, (ii) the agent has a financial stake in the event she is predicting, and (iii) the agent can hedge her prediction by taking an additional action. In contrast with previous literature, the PSR biases are characterized for all PSRs and all risk averse agents. Our results reveal complex distortions of reported beliefs, thereby raising concerns about the ability of PSRs to recover truthful beliefs in general decision-making environments.

(This abstract was borrowed from another version of this item.)

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL:
File Function: Forthcoming in European Economic Review
Download Restriction: no

Paper provided by Toulouse School of Economics (TSE) in its series TSE Working Papers with number 10-213.

in new window

Date of creation: Dec 2010
Date of revision:
Publication status: Published in European Economic Review, vol. 62, Elsevier, 2013, p. 17-40.
Handle: RePEc:tse:wpaper:23866
Contact details of provider: Phone: (+33) 5 61 12 86 23
Web page:

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Alan S. Blinder & Michael Ehrmann & Marcel Fratzscher & Jakob de Haan & David-Jan Jansen, 2008. "Central Bank Communication and Monetary Policy: A Survey of Theory and Evidence," DNB Working Papers 170, Netherlands Central Bank, Research Department.
  2. Sujoy Mukerji & Peter Klibanoff, 2002. "A Smooth Model of Decision,Making Under Ambiguity," Economics Series Working Papers 113, University of Oxford, Department of Economics.
  3. Palfrey, Thomas R. & Wang, Stephanie W., 2009. "On eliciting beliefs in strategic games," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 98-109, August.
  4. Noussair, C.N. & Trautmann, S.T. & van de Kuilen, G., 2011. "Higher Order Risk Attitudes, Demographics, and Financial Decisions," Discussion Paper 2011-055, Tilburg University, Center for Economic Research.
  5. von Gaudecker, H.M. & van Soest, A.H.O. & Wengstrom, E., 2009. "Heterogeneity in Risky Choice Behavior in a Broad Population," Discussion Paper 2009-12, Tilburg University, Center for Economic Research.
  6. Franklin Allen, 1987. "Notes--Discovering Personal Probabilities When Utility Functions are Unknown," Management Science, INFORMS, vol. 33(4), pages 542-544, April.
  7. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," Review of Economic Studies, Oxford University Press, vol. 75(3), pages 729-762.
  8. Urs Fischbacher & Simon Gaechter, 2009. "Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Good Experiments," Discussion Papers 2009-04, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
  9. Croson, Rachel T. A., 2000. "Thinking like a game theorist: factors affecting the frequency of equilibrium play," Journal of Economic Behavior & Organization, Elsevier, vol. 41(3), pages 299-314, March.
  10. Reinhard Selten & Abdolkarim Sadrieh & Klaus Abbink, 1999. "Money Does Not Induce Risk Neutral Behavior, but Binary Lotteries Do even Worse," Theory and Decision, Springer, vol. 46(3), pages 213-252, June.
  11. Michael Abramowicz, 2007. "The Hidden Beauty of the Quadratic Market Scoring Rule: A Uniform Liquidity Market Maker, with Variations," Journal of Prediction Markets, University of Buckingham Press, vol. 1(2), pages 111-125, July.
  12. Davis, Douglas D. & Holt, Charles a., 1993. "Experimental economics: Methods, problems and promise," Estudios Económicos, El Colegio de México, Centro de Estudios Económicos, vol. 8(2), pages 179-212.
  13. David Danz & Dietmar Fehr & Dorothea Kübler, 2012. "Information and beliefs in a repeated normal-form game," Experimental Economics, Springer, vol. 15(4), pages 622-640, December.
  14. Osband, Kent, 1989. "Optimal Forecasting Incentives," Journal of Political Economy, University of Chicago Press, vol. 97(5), pages 1091-1112, October.
  15. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
  16. Stefan T. Trautmann & Gijs Kuilen, 2015. "Belief Elicitation: A Horse Race among Truth Serums," Economic Journal, Royal Economic Society, vol. 125(589), pages 2116-2135, December.
  17. Daniel Read, 2005. "Monetary incentives, what are they good for?," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 265-276.
  18. Andersen, Steffen & Fountain, John & Harrison, Glenn W. & Rutström, Elisabet E., 2009. "Estimating Subjective Probabilities," Working Papers 05-2009, Copenhagen Business School, Department of Economics.
  19. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
  20. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, 09.
  21. Drazen Prelec, 1998. "The Probability Weighting Function," Econometrica, Econometric Society, vol. 66(3), pages 497-528, May.
  22. Ledyard, John & Hanson, Robin & Ishikida, Takashi, 2009. "An experimental test of combinatorial information markets," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 182-189, February.
  23. Karni, Edi, 1999. "Elicitation of Subjective Probabilities When Preferences Are State-Dependent," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 479-86, May.
  24. Marco Ottaviani & Peter Norman Sørensen, 2007. "Outcome Manipulation in Corporate Prediction Markets," Journal of the European Economic Association, MIT Press, vol. 5(2-3), pages 554-563, 04-05.
  25. Yoram Halevy, 2007. "Ellsberg Revisited: An Experimental Study," Econometrica, Econometric Society, vol. 75(2), pages 503-536, 03.
  26. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
  27. Simon Gaechter & Elke Renner, 2006. "The Effects of (Incentivized) Belief Elicitation in Public Good Experiments," Discussion Papers 2006-16, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
  28. Olivier Armantier & Nicolas Treich, 2009. "Subjective Probabilities In Games: An Application To The Overbidding Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1079-1102, November.
  29. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, 03.
  30. Manski, Charles F., 2002. "Identification of decision rules in experiments on simple games of proposal and response," European Economic Review, Elsevier, vol. 46(4-5), pages 880-891, May.
  31. Terrance Hurley & Jason Shogren, 2005. "An Experimental Comparison of Induced and Elicited Beliefs," Journal of Risk and Uncertainty, Springer, vol. 30(2), pages 169-188, January.
  32. repec:pri:cepsud:161blinder is not listed on IDEAS
  33. Jaffray, Jean-Yves & Karni, Edi, 1999. "Elicitation of Subjective Probabilities When the Initial Endowment is Unobservable," Journal of Risk and Uncertainty, Springer, vol. 18(1), pages 5-20, April.
  34. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-39, November.
  35. Blanco, Mariana & Engelmann, Dirk & Koch, Alexander K. & Normann, Hans-Theo, 2008. "Belief Elicitation in Experiments: Is there a Hedging Problem?," IZA Discussion Papers 3517, Institute for the Study of Labor (IZA).
  36. Armantier, Olivier & Boly, Amadou, 2011. "A controlled field experiment on corruption," European Economic Review, Elsevier, vol. 55(8), pages 1072-1082.
  37. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," Review of Economic Studies, Oxford University Press, vol. 76(4), pages 1461-1489.
  38. Huck, Steffen & Weizsacker, Georg, 2002. "Do players correctly estimate what others do? : Evidence of conservatism in beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 47(1), pages 71-85, January.
  39. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:tse:wpaper:23866. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.