Advanced Search
MyIDEAS: Login to save this paper or follow this series

Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging

Contents:

Author Info

  • Armantier, Olivier
  • Treich, Nicolas

Abstract

Accurate measurements of probabilistic beliefs have become increasingly important both in practice and in academia. Introduced by statisticians in the 1950s to promote truthful reports in simple environments, Proper Scoring Rules (PSR) are now arguably the most popular incentivized mechanisms to elicit an agent's beliefs. This paper generalizes the analysis of PSR to richer environments relevant to economists. More speci cally, we combine theory and experiment to study how beliefs reported with a PSR may be biased when i) the PSR payments are increased, ii) the agent has a financial stake in the event she is predicting, and iii) the agent can hedge her prediction by taking an additional action. Our results reveal complex distortions of reported beliefs, thereby raising concerns about the ability of PSR to recover truthful beliefs in general economic environments.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.tse-fr.eu/images/doc/wp/bee/10-156.pdf
File Function: Full text
Download Restriction: no

Bibliographic Info

Paper provided by Toulouse School of Economics (TSE) in its series TSE Working Papers with number 10-156.

as in new window
Length:
Date of creation: 15 Apr 2010
Date of revision:
Publication status: Published in European Economic Review, vol.�62, Elsevier, 2013, p.�17-40.
Handle: RePEc:tse:wpaper:22664

Contact details of provider:
Phone: (+33) 5 61 12 86 23
Web page: http://www.tse-fr.eu/
More information through EDIRC

Related research

Keywords:

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Steffen Andersen & John Fountain & Glenn W. Harrison & E. Elisabet Rutström, 2010. "Estimating Subjective Probabilities," Experimental Economics Center Working Paper Series 2010-08, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
  2. Drazen Prelec, 1998. "The Probability Weighting Function," Econometrica, Econometric Society, vol. 66(3), pages 497-528, May.
  3. Trautmann, S.T. & Kuilen, G. van de, 2011. "Belief Elicitation: A Horse Race among Truth Serums," Discussion Paper 2011-117, Tilburg University, Center for Economic Research.
  4. Franklin Allen, 1987. "Notes--Discovering Personal Probabilities When Utility Functions are Unknown," Management Science, INFORMS, vol. 33(4), pages 542-544, April.
  5. Palfrey, Thomas R. & Wang, Stephanie W., . "On eliciting beliefs in strategic games," Working Papers 1271, California Institute of Technology, Division of the Humanities and Social Sciences.
  6. Alan S. Blinder & Michael Ehrmann & Marcel Fratzscher & Jakob De Haan & David-Jan Jansen, 2008. "Central Bank Communication and Monetary Policy: A Survey of Theory and Evidence," Journal of Economic Literature, American Economic Association, vol. 46(4), pages 910-45, December.
  7. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
  8. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
  9. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, 09.
  10. Karni, Edi, 1999. "Elicitation of Subjective Probabilities When Preferences Are State-Dependent," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 479-86, May.
  11. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
  12. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer, vol. 13(3), pages 364-377, September.
  13. Peter Klibanoff & Massimo Marinacci & Sujoy Mukerji, 2002. "A smooth model of decision making under ambiguity," ICER Working Papers - Applied Mathematics Series 11-2003, ICER - International Centre for Economic Research, revised Apr 2003.
  14. Yoram Halevy, 2007. "Ellsberg Revisited: An Experimental Study," Econometrica, Econometric Society, vol. 75(2), pages 503-536, 03.
  15. Charles N. Noussair & Stefan T. Trautmann & Gijs van de Kuilen, 2014. "Higher Order Risk Attitudes, Demographics, and Financial Decisions," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 325-355.
  16. Reinhard Selten & Abdolkarim Sadrieh & Klaus Abbink, 1999. "Money Does Not Induce Risk Neutral Behavior, but Binary Lotteries Do even Worse," Theory and Decision, Springer, vol. 46(3), pages 213-252, June.
  17. Manski, Charles F., 2002. "Identification of decision rules in experiments on simple games of proposal and response," European Economic Review, Elsevier, vol. 46(4-5), pages 880-891, May.
  18. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer, vol. 13(4), pages 412-438, December.
  19. Ledyard, John & Hanson, Robin & Ishikida, Takashi, 2009. "An experimental test of combinatorial information markets," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 182-189, February.
  20. Camerer, Colin F. & Hogarth, Robin M., 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Working Papers 1059, California Institute of Technology, Division of the Humanities and Social Sciences.
  21. Urs Fischbacher & Simon Gaechter, 2008. "Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Good Experiments," CESifo Working Paper Series 2491, CESifo Group Munich.
  22. Daniel Read, 2005. "Monetary incentives, what are they good for?," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 265-276.
  23. Miguel A. Costa-Gomes & Georg Weizsäcker, 2004. "Stated Beliefs and Play in Normal Form Games," Levine's Bibliography 122247000000000236, UCLA Department of Economics.
  24. Gaudecker, H.M. von & Soest, A.H.O. van & Wengstrom, E., 2011. "Heterogeneity in risky choice behavior in a broad population," Open Access publications from Tilburg University urn:nbn:nl:ui:12-3857699, Tilburg University.
  25. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, 03.
  26. Miguel A. Costa-Gomes & Georg Weizs�cker, 2008. "Stated Beliefs and Play in Normal-Form Games," Review of Economic Studies, Oxford University Press, vol. 75(3), pages 729-762.
  27. Olivier Armantier & Nicolas Treich, 2009. "Subjective Probabilities In Games: An Application To The Overbidding Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1079-1102, November.
  28. Osband, Kent, 1989. "Optimal Forecasting Incentives," Journal of Political Economy, University of Chicago Press, vol. 97(5), pages 1091-1112, October.
  29. Armantier, Olivier & Boly, Amadou, 2011. "A controlled field experiment on corruption," European Economic Review, Elsevier, vol. 55(8), pages 1072-1082.
  30. Terrance Hurley & Jason Shogren, 2005. "An Experimental Comparison of Induced and Elicited Beliefs," Journal of Risk and Uncertainty, Springer, vol. 30(2), pages 169-188, January.
  31. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
  32. Marco Ottaviani & Peter Norman Sørensen, 2007. "Outcome Manipulation in Corporate Prediction Markets," Journal of the European Economic Association, MIT Press, vol. 5(2-3), pages 554-563, 04-05.
  33. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," Review of Economic Studies, Oxford University Press, vol. 76(4), pages 1461-1489.
  34. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-39, November.
  35. Jaffray, Jean-Yves & Karni, Edi, 1999. "Elicitation of Subjective Probabilities When the Initial Endowment is Unobservable," Journal of Risk and Uncertainty, Springer, vol. 18(1), pages 5-20, April.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Charles Manski & Claudia Neri, 2013. "First- and Second-order Subjective Expectations in Strategic Decision-Making: Experimental Evidence," 2013 Meeting Papers 73, Society for Economic Dynamics.
  2. Matthew Wiswall & Basit Zafar, 2011. "Belief updating among college students: evidence from experimental variation in information," Staff Reports 516, Federal Reserve Bank of New York.
  3. Dreber, Anna & Ellingsen, Tore & Johannesson, Magnus & Rand, David, 2011. "Do People Care about Social Context? Framing Effects in Dictator Games," Working Paper Series in Economics and Finance 738, Stockholm School of Economics.
  4. Federica Alberti & Anna Conte & Kei Tsutsui, 2014. "Accuracy of proposers' beliefs in an allocation-type game," Jena Economic Research Papers 2014-002, Friedrich-Schiller-University Jena, Max-Planck-Institute of Economics.
  5. Olivier Armantier & Scott Nelson & Giorgio Topa & Wilbert van der Klaauw & Basit Zafar, 2012. "The price is right: updating of inflation expectations in a randomized price information experiment," Staff Reports 543, Federal Reserve Bank of New York.
  6. Guy Mayraz, 2011. "Wishful Thinking," CEP Discussion Papers dp1092, Centre for Economic Performance, LSE.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:tse:wpaper:22664. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ().

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.