IDEAS home Printed from https://ideas.repec.org/p/jgu/wpaper/1818.html
   My bibliography  Save this paper

Belief Elicitation with Multiple Point Predictions

Author

Listed:
  • Markus Eyting

    () (Johannes Gutenberg-University)

  • Patrick Schmidt

    () (HITS gGmbH Heidelberg)

Abstract

We consider beliefs about real-valued outcomes, and show how to elicit the entire subjective probability distribution with binarized scoring rules. Further, we propose a simple, incentive compatible elicitation mechanism - multiple point predictions - that partially identifies the subjective probability distribution. Simultaneously eliciting multiple point predictions with linear incentives reveals the subjective probability distribution without pre-defined anchors or probabilistic statements. In a laboratory experiment, we test belief elicitation with multiple point predictions and compare it to the standard approach of eliciting discrete probabilities on pre-defined intervals. We find that elicitation with point predictions is faster, more convenient and more predictive of subsequent behavior. In the absence of anchors, the elicited distributions show evidence of first order biases, but are less prone to overconfidence. Further, the distributions are less accurate for uninformed participants, and more accurate under heterogeneous information.

Suggested Citation

  • Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 188, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
  • Handle: RePEc:jgu:wpaper:1818
    as

    Download full text from publisher

    File URL: https://download.uni-mainz.de/RePEc/pdf/Discussion_Paper_1818.pdf
    File Function: First version, 2018
    Download Restriction: no

    References listed on IDEAS

    as
    1. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    2. Xiangyu Qu, 2012. "A mechanism for eliciting a probability distribution," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-01437544, HAL.
    3. Charness, Gary & Dufwenberg, Martin, 2003. "Promises & Partnership," Research Papers in Economics 2003:3, Stockholm University, Department of Economics.
    4. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    5. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    6. James E. Matheson & Robert L. Winkler, 1976. "Scoring Rules for Continuous Probability Distributions," Management Science, INFORMS, vol. 22(10), pages 1087-1096, June.
    7. Michael Hurd & Maarten Van Rooij & Joachim Winter, 2011. "Stock market expectations of Dutch households," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 416-436, April.
    8. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," Review of Economic Studies, Oxford University Press, vol. 76(4), pages 1461-1489.
    9. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2014. "Eliciting subjective probabilities with binary lotteries," Journal of Economic Behavior & Organization, Elsevier, vol. 101(C), pages 128-140.
    10. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    11. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    12. Claudia Neri, 2015. "Eliciting beliefs in continuous-choice games: a double auction experiment," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 569-608, December.
    13. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, March.
    14. Craig R. Fox & Robert T. Clemen, 2005. "Subjective Probability Assessment in Decision Analysis: Partition Dependence and Bias Toward the Ignorance Prior," Management Science, INFORMS, vol. 51(9), pages 1417-1432, September.
    15. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," Review of Economic Studies, Oxford University Press, vol. 80(3), pages 984-1001.
    16. Fabian Gouret & Guillaume Hollard, 2011. "When Kahneman meets Manski: Using dual systems of reasoning to interpret subjective expectations of equity returns," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 371-392, April.
    17. Katja Kaufmann & Luigi Pistaferri, 2009. "Disentangling Insurance and Information in Intertemporal Consumption Choices," American Economic Review, American Economic Association, vol. 99(2), pages 387-392, May.
    18. Orazio Attanasio & Britta Augsburg, 2016. "Subjective Expectations and Income Processes in Rural India," Economica, London School of Economics and Political Science, vol. 83(331), pages 416-442, July.
    19. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    20. Qu, Xiangyu, 2012. "A mechanism for eliciting a probability distribution," Economics Letters, Elsevier, vol. 115(3), pages 399-400.
    21. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    22. Demuynck, Thomas, 2013. "A mechanism for eliciting the mean and quantiles of a random variable," Economics Letters, Elsevier, vol. 121(1), pages 121-123.
    23. Kristin J. Kleinjans & Arthur Van Soest, 2014. "Rounding, Focal Point Answers And Nonresponse To Subjective Probability Questions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 567-585, June.
    24. Gary Charness & Martin Dufwenberg, 2006. "Promises and Partnership," Econometrica, Econometric Society, vol. 74(6), pages 1579-1601, November.
    25. Palfrey, Thomas R. & Wang, Stephanie W., 2009. "On eliciting beliefs in strategic games," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 98-109, August.
    26. Fabio Bellini & Valeria Bignozzi, 2015. "On elicitable risk measures," Quantitative Finance, Taylor & Francis Journals, vol. 15(5), pages 725-733, May.
    27. Dominitz, Jeff, 2001. "Estimation of income expectations models using expectations and realization data," Journal of Econometrics, Elsevier, vol. 102(2), pages 165-195, June.
    28. Uriel Haran & Don A. Moore & Carey K. Morewedge, 2010. "A simple remedy for overprecision in judgment," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(7), pages 467-476, December.
    29. McKenzie, David & Gibson, John & Stillman, Steven, 2013. "A land of milk and honey with streets paved with gold: Do emigrants have over-optimistic expectations about incomes abroad?," Journal of Development Economics, Elsevier, vol. 102(C), pages 116-127.
    30. repec:wly:emetrp:v:84:y:2016:i::p:1983-2027 is not listed on IDEAS
    31. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    32. Fabian Gouret & Guillaume Hollard, 2011. "When Kahneman meets Manski: Using dual systems of reasoning to interpret subjective expectations of equity returns," Post-Print halshs-00867700, HAL.
    33. Adeline Delavande & Xavier Giné & David McKenzie, 2011. "Eliciting probabilistic expectations with visual aids in developing countries: how sensitive are answers to variations in elicitation design?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 479-497, April.
    34. Dufwenberg, Martin & Gneezy, Uri, 2000. "Measuring Beliefs in an Experimental Lost Wallet Game," Games and Economic Behavior, Elsevier, vol. 30(2), pages 163-182, February.
    35. Manski, Charles F. & Neri, Claudia, 2013. "First- and second-order subjective expectations in strategic decision-making: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 81(C), pages 232-254.
    36. Ruth Vargas Hill, 2010. "Liberalisation and Producer Price Risk: Examining Subjective Expectations in the Ugandan Coffee Market," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 19(4), pages 433-458, August.
    37. repec:taf:jnlbes:v:36:y:2018:i:3:p:505-515 is not listed on IDEAS
    38. L. Bissonnette & J. de Bresser, 2018. "Eliciting Subjective Survival Curves: Lessons from Partial Identification," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(3), pages 505-515, July.
    39. Efe A. Ok, 2007. "Preliminaries of Real Analysis, from Real Analysis with Economic Applications," Introductory Chapters,in: Real Analysis with Economic Applications Princeton University Press.
    40. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    41. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    42. Adeline Delavande, 2008. "Measuring revisions to subjective expectations," Journal of Risk and Uncertainty, Springer, vol. 36(1), pages 43-82, February.
    43. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    44. repec:taf:jnlbes:v:30:y:2012:i:1:p:125-131 is not listed on IDEAS
    45. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-1339, November.
    46. Delavande, Adeline & Giné, Xavier & McKenzie, David, 2011. "Measuring subjective expectations in developing countries: A critical review and new evidence," Journal of Development Economics, Elsevier, vol. 94(2), pages 151-163, March.
    47. Gianna Boero & Jeremy Smith & Kenneth F. Wallis, 2015. "The Measurement and Characteristics of Professional Forecasters' Uncertainty," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1029-1046, November.
    48. Charles Bellemare & Luc Bissonnette & Sabine Kröger, 2011. "Flexible Approximation of Subjective Expectations Using Probability Questions," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 125-131, April.
    49. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    50. Garthwaite, Paul H. & Kadane, Joseph B. & O'Hagan, Anthony, 2005. "Statistical Methods for Eliciting Probability Distributions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 680-701, June.
    51. Lahiri, Kajal & Teigland, Christie, 1987. "On the normality of probability distributions of inflation and GNP forecasts," International Journal of Forecasting, Elsevier, vol. 3(2), pages 269-279.
    52. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd & Ulm, Eric R., 2017. "Scoring rules for subjective probability distributions," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 430-448.
    53. Guiso, Luigi & Jappelli, Tullio & Pistaferri, Luigi, 2002. "An Empirical Analysis of Earnings and Employment Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 241-253, April.
    54. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    55. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    56. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Schmidt, 2019. "Elicitation of ambiguous beliefs with mixing bets," Papers 1902.07447, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jgu:wpaper:1818. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Research Unit IPP). General contact details of provider: http://edirc.repec.org/data/vlmaide.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.