IDEAS home Printed from https://ideas.repec.org/p/jgu/wpaper/1818.html
   My bibliography  Save this paper

Belief Elicitation with Multiple Point Predictions

Author

Listed:
  • Markus Eyting

    () (Johannes Gutenberg-University)

  • Patrick Schmidt

    () (HITS gGmbH Heidelberg)

Abstract

We propose a simple, incentive compatible elicitation mechanism to elicit beliefs about real-valued outcomes - multiple point predictions. Simultaneously eliciting multiple point predictions with linear incentives reveals the subjective probability distribution without pre-defined intervals or probabilistic statements. We show that the approach is theoretically as robust as existing methods, while adapting flexibly to heterogeneous beliefs. In a laboratory experiment, we compare our method to the standard approach of eliciting discrete probabilities on pre-defined intervals. We find that elicitation with multiple point predictions is faster, more convenient and more predictive of subsequent behavior. We further find that multiple point predictions are more accurate if participants have heterogeneous beliefs. Finally, we provide experimental evidence that pre-defined intervals anchor reports.

Suggested Citation

  • Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 17 May 2020.
  • Handle: RePEc:jgu:wpaper:1818
    as

    Download full text from publisher

    File URL: https://download.uni-mainz.de/RePEc/pdf/Discussion_Paper_1818.pdf
    File Function: First version, 2018
    Download Restriction: no

    References listed on IDEAS

    as
    1. Adeline Delavande & Xavier Giné & David McKenzie, 2011. "Eliciting probabilistic expectations with visual aids in developing countries: how sensitive are answers to variations in elicitation design?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 479-497, April.
    2. Giordani, Paolo & Soderlind, Paul, 2003. "Inflation forecast uncertainty," European Economic Review, Elsevier, vol. 47(6), pages 1037-1059, December.
    3. Dufwenberg, Martin & Gneezy, Uri, 2000. "Measuring Beliefs in an Experimental Lost Wallet Game," Games and Economic Behavior, Elsevier, vol. 30(2), pages 163-182, February.
    4. Manski, Charles F. & Neri, Claudia, 2013. "First- and second-order subjective expectations in strategic decision-making: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 81(C), pages 232-254.
    5. Qu, Xiangyu, 2012. "A mechanism for eliciting a probability distribution," Economics Letters, Elsevier, vol. 115(3), pages 399-400.
    6. Charness, Gary & Dufwenberg, Martin, 2003. "Promises & Partnership," Research Papers in Economics 2003:3, Stockholm University, Department of Economics.
    7. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    8. Ruth Vargas Hill, 2010. "Liberalisation and Producer Price Risk: Examining Subjective Expectations in the Ugandan Coffee Market," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 19(4), pages 433-458, August.
    9. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    10. James E. Matheson & Robert L. Winkler, 1976. "Scoring Rules for Continuous Probability Distributions," Management Science, INFORMS, vol. 22(10), pages 1087-1096, June.
    11. L. Bissonnette & J. de Bresser, 2018. "Eliciting Subjective Survival Curves: Lessons from Partial Identification," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(3), pages 505-515, July.
    12. Palfrey, Thomas R. & Wang, Stephanie W., 2009. "On eliciting beliefs in strategic games," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 98-109, August.
    13. Michael Hurd & Maarten Van Rooij & Joachim Winter, 2011. "Stock market expectations of Dutch households," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 416-436, April.
    14. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," Review of Economic Studies, Oxford University Press, vol. 76(4), pages 1461-1489.
    15. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2014. "Eliciting subjective probabilities with binary lotteries," Journal of Economic Behavior & Organization, Elsevier, vol. 101(C), pages 128-140.
    16. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    17. Efe A. Ok, 2007. "Preliminaries of Real Analysis, from Real Analysis with Economic Applications," Introductory Chapters, in: Real Analysis with Economic Applications, Princeton University Press.
    18. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    19. Lahiri, Kajal & Teigland, Christie & Zaporowski, Mark, 1988. "Interest Rates and the Subjective Probability Distribution of Inflation Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 20(2), pages 233-248, May.
    20. Claudia Neri, 2015. "Eliciting beliefs in continuous-choice games: a double auction experiment," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 569-608, December.
    21. Tilmann Gneiting & Fadoua Balabdaoui & Adrian E. Raftery, 2007. "Probabilistic forecasts, calibration and sharpness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 243-268, April.
    22. McKenzie, David & Gibson, John & Stillman, Steven, 2013. "A land of milk and honey with streets paved with gold: Do emigrants have over-optimistic expectations about incomes abroad?," Journal of Development Economics, Elsevier, vol. 102(C), pages 116-127.
    23. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, March.
    24. Craig R. Fox & Robert T. Clemen, 2005. "Subjective Probability Assessment in Decision Analysis: Partition Dependence and Bias Toward the Ignorance Prior," Management Science, INFORMS, vol. 51(9), pages 1417-1432, September.
    25. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," Review of Economic Studies, Oxford University Press, vol. 80(3), pages 984-1001.
    26. Fabian Gouret & Guillaume Hollard, 2011. "When Kahneman meets Manski: Using dual systems of reasoning to interpret subjective expectations of equity returns," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 371-392, April.
    27. Katja Kaufmann & Luigi Pistaferri, 2009. "Disentangling Insurance and Information in Intertemporal Consumption Choices," American Economic Review, American Economic Association, vol. 99(2), pages 387-392, May.
    28. Adeline Delavande, 2008. "Measuring revisions to subjective expectations," Journal of Risk and Uncertainty, Springer, vol. 36(1), pages 43-82, February.
    29. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    30. Orazio Attanasio & Britta Augsburg, 2016. "Subjective Expectations and Income Processes in Rural India," Economica, London School of Economics and Political Science, vol. 83(331), pages 416-442, July.
    31. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    32. repec:taf:jnlbes:v:30:y:2012:i:1:p:125-131 is not listed on IDEAS
    33. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-1339, November.
    34. Demuynck, Thomas, 2013. "A mechanism for eliciting the mean and quantiles of a random variable," Economics Letters, Elsevier, vol. 121(1), pages 121-123.
    35. Delavande, Adeline & Giné, Xavier & McKenzie, David, 2011. "Measuring subjective expectations in developing countries: A critical review and new evidence," Journal of Development Economics, Elsevier, vol. 94(2), pages 151-163, March.
    36. Gianna Boero & Jeremy Smith & Kenneth F. Wallis, 2015. "The Measurement and Characteristics of Professional Forecasters' Uncertainty," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(7), pages 1029-1046, November.
    37. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    38. Charles Bellemare & Luc Bissonnette & Sabine Kröger, 2011. "Flexible Approximation of Subjective Expectations Using Probability Questions," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 125-131, April.
    39. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    40. Garthwaite, Paul H. & Kadane, Joseph B. & O'Hagan, Anthony, 2005. "Statistical Methods for Eliciting Probability Distributions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 680-701, June.
    41. Kristin J. Kleinjans & Arthur Van Soest, 2014. "Rounding, Focal Point Answers And Nonresponse To Subjective Probability Questions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 567-585, June.
    42. Gary Charness & Martin Dufwenberg, 2006. "Promises and Partnership," Econometrica, Econometric Society, vol. 74(6), pages 1579-1601, November.
    43. Lahiri, Kajal & Teigland, Christie, 1987. "On the normality of probability distributions of inflation and GNP forecasts," International Journal of Forecasting, Elsevier, vol. 3(2), pages 269-279.
    44. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd & Ulm, Eric R., 2017. "Scoring rules for subjective probability distributions," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 430-448.
    45. Guiso, Luigi & Jappelli, Tullio & Pistaferri, Luigi, 2002. "An Empirical Analysis of Earnings and Employment Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 241-253, April.
    46. Fabio Bellini & Valeria Bignozzi, 2015. "On elicitable risk measures," Quantitative Finance, Taylor & Francis Journals, vol. 15(5), pages 725-733, May.
    47. Dominitz, Jeff, 2001. "Estimation of income expectations models using expectations and realization data," Journal of Econometrics, Elsevier, vol. 102(2), pages 165-195, June.
    48. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    49. Uriel Haran & Don A. Moore & Carey K. Morewedge, 2010. "A simple remedy for overprecision in judgment," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(7), pages 467-476, December.
    50. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    51. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    52. Jay Lu, 2016. "Random Choice and Private Information," Econometrica, Econometric Society, vol. 84, pages 1983-2027, November.
    53. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Schmidt, 2019. "Elicitation of ambiguous beliefs with mixing bets," Papers 1902.07447, arXiv.org, revised Oct 2019.

    More about this item

    Keywords

    elicitation of subjective expectations; partial identification; quantiles; overconfidence; experiment;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jgu:wpaper:1818. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Research Unit IPP). General contact details of provider: http://edirc.repec.org/data/vlmaide.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.