IDEAS home Printed from https://ideas.repec.org/a/wly/econjl/v125y2015i589p2116-2135.html
   My bibliography  Save this article

Belief Elicitation: A Horse Race among Truth Serums

Author

Listed:
  • Stefan T. Trautmann
  • Gijs Kuilen

Abstract

In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Stefan T. Trautmann & Gijs Kuilen, 2015. "Belief Elicitation: A Horse Race among Truth Serums," Economic Journal, Royal Economic Society, vol. 125(589), pages 2116-2135, December.
    • Handle: RePEc:wly:econjl:v:125:y:2015:i:589:p:2116-2135
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/ecoj.2015.125.issue-589
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Mohammed Abdellaoui & Frank Vossmann & Martin Weber, 2005. "Choice-Based Elicitation and Decomposition of Decision Weights for Gains and Losses Under Uncertainty," Management Science, INFORMS, vol. 51(9), pages 1384-1399, September.
    2. Lisa Cameron & Manisha Shah, 2015. "Risk-Taking Behavior in the Wake of Natural Disasters," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 484-515.
    3. Antonio Cabrales & Raffaele Miniaci & Marco Piovesan & Giovanni Ponti, 2010. "Social Preferences and Strategic Uncertainty: An Experiment on Markets and Contracts," American Economic Review, American Economic Association, vol. 100(5), pages 2261-2278, December.
    4. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    5. Luigi Guiso & Giuseppe Parigi, 1999. "Investment and Demand Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 185-227.
    6. Ernst Fehr & Klaus M. Schmidt, 1999. "A Theory of Fairness, Competition, and Cooperation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(3), pages 817-868.
    7. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2010. "Subjective beliefs formation and elicitation rules: experimental evidence," Post-Print halshs-00543828, HAL.
    8. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    9. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    10. Huck, Steffen & Weizsacker, Georg, 2002. "Do players correctly estimate what others do? : Evidence of conservatism in beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 47(1), pages 71-85, January.
    11. Ferdinand Vieider, 2011. "Separating real incentives and accountability," Experimental Economics, Springer;Economic Science Association, vol. 14(4), pages 507-518, November.
    12. Guiso, Luigi & Jappelli, Tullio & Terlizzese, Daniele, 1992. "Earnings uncertainty and precautionary saving," Journal of Monetary Economics, Elsevier, vol. 30(2), pages 307-337, November.
    13. Frank Heinemann & Rosemarie Nagel & Peter Ockenfels, 2009. "Measuring Strategic Uncertainty in Coordination Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(1), pages 181-221.
    14. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    15. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    16. Ernst Fehr & Klaus M. Schmidt, "undated". "Theories of Fairness and Reciprocity - Evidence and Economic Applications," IEW - Working Papers 075, Institute for Empirical Research in Economics - University of Zurich.
    17. Charles Bellemare & Sabine Kröger & Arthur van Soest, 2008. "Measuring Inequity Aversion in a Heterogeneous Population Using Experimental Decisions and Subjective Probabilities," Econometrica, Econometric Society, vol. 76(4), pages 815-839, July.
    18. Shiller, Robert J & Kon-Ya, Fumiko & Tsutsui, Yoshiro, 1996. "Why Did the Nikkei Crash? Expanding the Scope of Expectations Data Collection," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 156-164, February.
    19. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    20. repec:cup:cbooks:9780521818728 is not listed on IDEAS
    21. Orazio P. Attanasio, 2009. "Expectations and Perceptions in Developing Countries: Their Measurement and Their Use," American Economic Review, American Economic Association, vol. 99(2), pages 87-92, May.
    22. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    23. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    24. William A. Branch, 2004. "The Theory of Rationally Heterogeneous Expectations: Evidence from Survey Data on Inflation Expectations," Economic Journal, Royal Economic Society, vol. 114(497), pages 592-621, July.
    25. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    26. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    27. Li Hao & Daniel Houser, 2010. "Getting It Right the First Time: Belief Elicitation with Novice Participants," Working Papers 1015, George Mason University, Interdisciplinary Center for Economic Science.
    28. Khwaja, Ahmed & Sloan, Frank & Salm, Martin, 2006. "Evidence on preferences and subjective beliefs of risk takers: The case of smokers," International Journal of Industrial Organization, Elsevier, vol. 24(4), pages 667-682, July.
    29. Carman, K.G. & Kooreman, P., 2010. "Flu Shots, Mammogram, and the Perception of Probabilities," Other publications TiSEM fba970b8-6fc7-449b-acf9-9, Tilburg University, School of Economics and Management.
    30. Rey-Biel, Pedro, 2009. "Equilibrium play and best response to (stated) beliefs in normal form games," Games and Economic Behavior, Elsevier, vol. 65(2), pages 572-585, March.
    31. Delavande, Adeline & Giné, Xavier & McKenzie, David, 2011. "Measuring subjective expectations in developing countries: A critical review and new evidence," Journal of Development Economics, Elsevier, vol. 94(2), pages 151-163, March.
    32. repec:cup:cbooks:9780521524131 is not listed on IDEAS
    33. Richard H. Thaler & Eric J. Johnson, 1990. "Gambling with the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice," Management Science, INFORMS, vol. 36(6), pages 643-660, June.
    34. repec:cup:cbooks:9780521818735 is not listed on IDEAS
    35. repec:cup:cbooks:9780521524124 is not listed on IDEAS
    36. Michael D. Hurd, 2009. "Subjective Probabilities in Household Surveys," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 543-564, May.
    37. Hessel Oosterbeek & Randolph Sloof & Gijs van de Kuilen, 2004. "Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis," Experimental Economics, Springer;Economic Science Association, vol. 7(2), pages 171-188, June.
    38. Ronald Bosman & Frans van Winden, 2002. "Emotional Hazard in a Power-to-take Experiment," Economic Journal, Royal Economic Society, vol. 112(476), pages 147-169, January.
    39. Machina, Mark J & Schmeidler, David, 1992. "A More Robust Definition of Subjective Probability," Econometrica, Econometric Society, vol. 60(4), pages 745-780, July.
    40. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    41. Chris Starmer, 2000. "Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 332-382, June.
    42. repec:cup:cbooks:9780521818742 is not listed on IDEAS
    43. repec:cup:cbooks:9780521524117 is not listed on IDEAS
    44. Wei Li, 2007. "Changing One's Mind when the Facts Change: Incentives of Experts and the Design of Reporting Protocols," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1175-1194.
    45. Olivier Armantier & Nicolas Treich, 2009. "Subjective Probabilities In Games: An Application To The Overbidding Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1079-1102, November.
    46. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    2. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    3. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    4. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    5. Dominik Bauer & Irenaeus Wolff, 2018. "Biases in Beliefs: Experimental Evidence," TWI Research Paper Series 109, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    6. Chen Li & Uyanga Turmunkh & Peter P. Wakker, 2019. "Trust as a decision under ambiguity," Experimental Economics, Springer;Economic Science Association, vol. 22(1), pages 51-75, March.
    7. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    8. Bauer, Dominik & Wolff, Irenaeus, 2019. "Biases in Beliefs," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203601, Verein für Socialpolitik / German Economic Association.
    9. Theo Offerman & Asa B. Palley, 2016. "Lossed in translation: an off-the-shelf method to recover probabilistic beliefs from loss-averse agents," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 1-30, March.
    10. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    11. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    12. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    13. Popescu, Andreea Victoria, 2020. "Essays in asset pricing and auctions," Other publications TiSEM 879f7643-7123-4bc8-a5e7-6, Tilburg University, School of Economics and Management.
    14. Adriana Breaban & Charles N. Noussair & Andreea Victoria Popescu, 2018. "Your money or your time? Experimental evidence on overbidding in all-pay auctions," Working Papers 18-20, Chapman University, Economic Science Institute.
    15. Costa-Gomes, Miguel A. & Huck, Steffen & Weizsäcker, Georg, 2014. "Beliefs and actions in the trust game: Creating instrumental variables to estimate the causal effect," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 88, pages 298-309.
    16. Claudia Neri, 2015. "Eliciting beliefs in continuous-choice games: a double auction experiment," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 569-608, December.
    17. Olivier L'Haridon & Craig S. Webb & Horst Zank, 2021. "An Effective and Simple Tool for Measuring Loss Aversion," Economics Discussion Paper Series 2107, Economics, The University of Manchester.
    18. Sandra Ludwig & Julia Nafziger, 2011. "Beliefs about overconfidence," Theory and Decision, Springer, vol. 70(4), pages 475-500, April.
    19. Polonio, Luca & Coricelli, Giorgio, 2019. "Testing the level of consistency between choices and beliefs in games using eye-tracking," Games and Economic Behavior, Elsevier, vol. 113(C), pages 566-586.
    20. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.

    More about this item

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:econjl:v:125:y:2015:i:589:p:2116-2135. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/resssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.