IDEAS home Printed from https://ideas.repec.org/a/kap/jrisku/v62y2021i2d10.1007_s11166-021-09349-6.html
   My bibliography  Save this article

Simple belief elicitation: An experimental evaluation

Author

Listed:
  • Karl Schlag

    (University of Vienna)

  • James Tremewan

    (University of Auckland)

Abstract

We present a method for eliciting beliefs about probabilities when multiple realisations of an outcome are available, the “frequency” method. The method is applicable for any reasonable utility function. Unlike existing techniques that account for deviations from risk-neutrality, this method is highly transparent to subjects and easy to implement. Rather than identifying point beliefs these methods identify bounds on beliefs, thus trading off precision for generality and simplicity. An experimental comparison of this method and a popular alternative, the Karni method, shows that subjects indeed find the frequency method easier to understand. Significantly, we show that confusion due to the complexity of the Karni method leads to less cognitively able subjects erroneously stating a belief of 50%, a bias not present in the frequency method.

Suggested Citation

  • Karl Schlag & James Tremewan, 2021. "Simple belief elicitation: An experimental evaluation," Journal of Risk and Uncertainty, Springer, vol. 62(2), pages 137-155, April.
    • Handle: RePEc:kap:jrisku:v:62:y:2021:i:2:d:10.1007_s11166-021-09349-6
    DOI: 10.1007/s11166-021-09349-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11166-021-09349-6
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11166-021-09349-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Terrance Hurley & Jason Shogren, 2005. "An Experimental Comparison of Induced and Elicited Beliefs," Journal of Risk and Uncertainty, Springer, vol. 30(2), pages 169-188, January.
    2. Qu, Xiangyu, 2012. "A mechanism for eliciting a probability distribution," Economics Letters, Elsevier, vol. 115(3), pages 399-400.
    3. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    4. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    5. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    6. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, March.
    7. Ingrid Burfurd & Tom Wilkening, 2018. "Experimental guidance for eliciting beliefs with the Stochastic Becker–DeGroot–Marschak mechanism," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 4(1), pages 15-28, July.
    8. Karl H.Schlag, 2015. "Who gives Direction to Statistical Testing? Best Practice meets Mathematically Correct Tests," Vienna Economics Papers vie1512, University of Vienna, Department of Economics.
    9. Nathaniel T. Wilcox & Nick Feltovich, 2000. "Thinking Like a Game Theorist: Comment," Monash Economics Working Papers archive-30, Monash University, Department of Economics.
    10. Le Coq, Chloé & Tremewan, James & Wagner, Alexander K., 2015. "On the effects of group identity in strategic environments," European Economic Review, Elsevier, vol. 76(C), pages 239-252.
    11. Bhatt, Meghana & Camerer, Colin F., 2005. "Self-referential thinking and equilibrium as states of mind in games: fMRI evidence," Games and Economic Behavior, Elsevier, vol. 52(2), pages 424-459, August.
    12. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    13. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    14. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    15. Karl H.Schlag, 2015. "Who gives Direction to Statistical Testing? Best Practice meets Mathematically Correct Tests," Vienna Economics Papers 1512, University of Vienna, Department of Economics.
    16. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    17. Karl Schlag, 2008. "A new method for constructing exact tests without making any assumptions," Economics Working Papers 1109, Department of Economics and Business, Universitat Pompeu Fabra.
    18. Isaac M. Lipkus & Greg Samsa & Barbara K. Rimer, 2001. "General Performance on a Numeracy Scale among Highly Educated Samples," Medical Decision Making, , vol. 21(1), pages 37-44, February.
    19. Li Hao & Daniel Houser, 2012. "Belief elicitation in the presence of naïve respondents: An experimental study," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 161-180, April.
    20. Marilyn M. Schapira & Ann B. Nattinger & Colleen A. McHorney, 2001. "Frequency or Probability? A Qualitative Study of Risk Communication Formats Used in Health Care," Medical Decision Making, , vol. 21(6), pages 459-467, December.
    21. Ingrid Burfurd & Tom Wilkening, 2022. "Cognitive heterogeneity and complex belief elicitation," Experimental Economics, Springer;Economic Science Association, vol. 25(2), pages 557-592, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elias Bouacida & Renaud Foucart, 2020. "The acceptability of lotteries in allocation problems," Working Papers 301646245, Lancaster University Management School, Economics Department.
    2. Ďuriník, Michal & Morita, Hodaka & Servátka, Maroš & Zhang, Le, 2023. "Promotions and Group Identity," MPRA Paper 119389, University Library of Munich, Germany.
    3. Haeckl, Simone, 2022. "Image concerns in ex-ante self-assessments–Gender differences and behavioral consequences," Labour Economics, Elsevier, vol. 76(C).
    4. Canen, Nathan & Chakraborty, Anujit, 2023. "Belief elicitation in political protest experiments: When the mode does not teach us about incentives to protest," Journal of Economic Behavior & Organization, Elsevier, vol. 216(C), pages 320-331.
    5. Kim, Duk Gyoo & Kim, Hee Chun, 2022. "Probability matching and strategic decision making," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    6. Elias Bouacida & Renaud Foucart, 2022. "Rituals of Reason," Working Papers 344119591, Lancaster University Management School, Economics Department.
    7. Lata Gangadharan & Philip J. Grossman & Nina Xue, 2022. "Stepping Stone: Identifying self-image concerns from motivated beliefs: Does it matter how and whom you ask?," Monash Economics Working Papers 2022-05, Monash University, Department of Economics.
    8. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    9. Lata Gangadharan & Philip J. Grossman & Nina Xue, 2021. "Identifying self-image concerns from motivated beliefs: Does it matter how and whom you ask?," Monash Economics Working Papers 2021-17, Monash University, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    2. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    3. Burdea, Valeria & Woon, Jonathan, 2022. "Online belief elicitation methods," Journal of Economic Psychology, Elsevier, vol. 90(C).
    4. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    5. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    6. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    7. Crosetto, Paolo & Filippin, Antonio & Katuščák, Peter & Smith, John, 2020. "Central tendency bias in belief elicitation," Journal of Economic Psychology, Elsevier, vol. 78(C).
    8. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    9. Kaushik Basu & Leonardo Becchetti & Luca Stanca, 2011. "Experiments with the Traveler’s Dilemma: welfare, strategic choice and implicit collusion," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 37(4), pages 575-595, October.
    10. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    11. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    12. Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 16 Nov 2020.
    13. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    14. Dominik Bauer & Irenaeus Wolff, 2018. "Biases in Beliefs: Experimental Evidence," TWI Research Paper Series 109, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    15. Bauer, Dominik & Wolff, Irenaeus, 2019. "Biases in Beliefs," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203601, Verein für Socialpolitik / German Economic Association.
    16. Sandra Ludwig & Julia Nafziger, 2011. "Beliefs about overconfidence," Theory and Decision, Springer, vol. 70(4), pages 475-500, April.
    17. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    18. Grewenig, Elisabeth & Lergetporer, Philipp & Werner, Katharina & Woessmann, Ludger, 2022. "Incentives, search engines, and the elicitation of subjective beliefs: Evidence from representative online survey experiments," Journal of Econometrics, Elsevier, vol. 231(1), pages 304-326.
    19. Tsakas, Elias, 2018. "Robust scoring rules," Research Memorandum 023, Maastricht University, Graduate School of Business and Economics (GSBE).
    20. Theo Offerman & Asa B. Palley, 2016. "Lossed in translation: an off-the-shelf method to recover probabilistic beliefs from loss-averse agents," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 1-30, March.

    More about this item

    Keywords

    Belief elicitation; Experiment; Frequency format; Cognitive ability;
    All these keywords.

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jrisku:v:62:y:2021:i:2:d:10.1007_s11166-021-09349-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.