IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/pj9u2.html
   My bibliography  Save this paper

The Risk Elicitation Puzzle Revisited: Across-Methods (In)consistency?

Author

Listed:
  • Holzmeister, Felix

    (University of Innsbruck)

  • Stefan, Matthias

Abstract

With the rise of experimental methods in the social sciences, numerous methods to elicit and classify people's risk attitudes in the laboratory have evolved. However, evidence suggests that people's attitudes towards risk may change considerably when measured with different methods. Based on a with-subject design using four widespread risk elicitation methods, we find that different procedures indeed give rise to considerably varying estimates of individual and aggregate level risk preferences. Conducting simulation exercises to obtain benchmarks for subjects' behavior, we find that the observed heterogeneity in risk preference estimates across methods is qualitatively similar to the heterogeneity arising from independent random draws from choices in the experimental tasks. Our study, however, provides evidence that subjects are surprisingly well aware of the variation in the riskiness of their choices. We argue that this calls into question the common interpretation of variation in revealed risk preferences being inconsistent.

Suggested Citation

  • Holzmeister, Felix & Stefan, Matthias, 2019. "The Risk Elicitation Puzzle Revisited: Across-Methods (In)consistency?," OSF Preprints pj9u2, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:pj9u2
    DOI: 10.31219/osf.io/pj9u2
    as

    Download full text from publisher

    File URL: https://osf.io/download/5c9dc810e0a7de0017970965/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/pj9u2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Tamás Csermely & Alexander Rabas, 2016. "How to reveal people’s preferences: Comparing time consistency and predictive power of multiple price list risk elicitation methods," Journal of Risk and Uncertainty, Springer, vol. 53(2), pages 107-136, December.
    2. Crosetto, P. & Filippin, A., 2017. "Safe options induce gender differences in risk attitudes," Working Papers 2017-05, Grenoble Applied Economics Laboratory (GAEL).
    3. Loomes, Graham & Moffatt, Peter G & Sugden, Robert, 2002. "A Microeconometric Test of Alternative Stochastic Theories of Risky Choice," Journal of Risk and Uncertainty, Springer, vol. 24(2), pages 103-130, March.
    4. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde, 2010. "Are Risk Aversion and Impatience Related to Cognitive Ability?," American Economic Review, American Economic Association, vol. 100(3), pages 1238-1260, June.
    5. Jose Apesteguia & Miguel A. Ballester, 2018. "Monotone Stochastic Choice Models: The Case of Risk and Time Preferences," Journal of Political Economy, University of Chicago Press, vol. 126(1), pages 74-106.
    6. David Bruner, 2011. "Multiple switching behaviour in multiple price lists," Applied Economics Letters, Taylor & Francis Journals, vol. 18(5), pages 417-420.
    7. Catherine C. Eckel & Philip J. Grossman, 2002. "Sex Differences and Statistical Stereotyping in Attitudes Toward Financial Risk," Monash Economics Working Papers archive-03, Monash University, Department of Economics.
    8. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    9. John Hey & Andrea Morone & Ulrich Schmidt, 2009. "Noise and bias in eliciting preferences," Journal of Risk and Uncertainty, Springer, vol. 39(3), pages 213-235, December.
    10. Louis Lévy-Garboua & Hela Maafi & David Masclet & Antoine Terracol, 2012. "Risk aversion and framing effects," Experimental Economics, Springer;Economic Science Association, vol. 15(1), pages 128-144, March.
    11. Andreas Pedroni & Renato Frey & Adrian Bruhin & Gilles Dutilh & Ralph Hertwig & Jörg Rieskamp, 2017. "The risk elicitation puzzle," Nature Human Behaviour, Nature, vol. 1(11), pages 803-809, November.
    12. Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
    13. Hans P. Binswanger, 1980. "Attitudes Toward Risk: Experimental Measurement in Rural India," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 62(3), pages 395-407.
    14. Friedman, Daniel & Habib, Sameh & James, Duncan & Crockett, Sean, 2018. "Varieties of risk elicitation," Discussion Papers, Research Professorship Market Design: Theory and Pragmatics SP II 2018-501, WZB Berlin Social Science Center.
    15. Jayson L. Lusk & Keith H. Coble, 2005. "Risk Perceptions, Risk Preference, and Acceptance of Risky Food," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(2), pages 393-405.
    16. Anderson, Lisa R. & Mellor, Jennifer M., 2008. "Predicting health behaviors with an experimental measure of risk preference," Journal of Health Economics, Elsevier, vol. 27(5), pages 1260-1274, September.
    17. Gul, Faruk, 1991. "A Theory of Disappointment Aversion," Econometrica, Econometric Society, vol. 59(3), pages 667-686, May.
    18. Lisa Anderson & Jennifer Mellor, 2009. "Are risk preferences stable? Comparing an experimental measure with a validated survey-based measure," Journal of Risk and Uncertainty, Springer, vol. 39(2), pages 137-160, October.
    19. Quang Nguyen & Colin Camerer & Tomomi Tanaka, 2010. "Risk and Time Preferences Linking Experimental and Household Data from Vietnam," Post-Print halshs-00547090, HAL.
    20. Dulleck Uwe & Fooken Jonas & Fell Jacob, 2015. "Within-Subject Intra- and Inter-Method Consistency of Two Experimental Risk Attitude Elicitation Methods," German Economic Review, De Gruyter, vol. 16(1), pages 104-121, February.
    21. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    22. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    23. Ola Andersson & Håkan J. Holm & Jean-Robert Tyran & Erik Wengström, 2016. "Risk Aversion Relates to Cognitive Ability: Preferences Or Noise?," Journal of the European Economic Association, European Economic Association, vol. 14(5), pages 1129-1154.
    24. Robin Cubitt & Chris Starmer & Robert Sugden, 1998. "On the Validity of the Random Lottery Incentive System," Experimental Economics, Springer;Economic Science Association, vol. 1(2), pages 115-131, September.
    25. John D. Hey & Chris Orme, 2018. "Investigating Generalizations Of Expected Utility Theory Using Experimental Data," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 3, pages 63-98, World Scientific Publishing Co. Pte. Ltd..
    26. Paolo Crosetto & Antonio Filippin, 2013. "The “bomb” risk elicitation task," Journal of Risk and Uncertainty, Springer, vol. 47(1), pages 31-65, August.
    27. Jehoshua Eliashberg & John R. Hauser, 1985. "A Measurement Error Approach for Modeling Consumer Risk Preference," Management Science, INFORMS, vol. 31(1), pages 1-25, January.
    28. Cohen, Michele & Jaffray, Jean-Yves & Said, Tanios, 1987. "Experimental comparison of individual behavior under risk and under uncertainty for gains and for losses," Organizational Behavior and Human Decision Processes, Elsevier, vol. 39(1), pages 1-22, February.
    29. Peter P. Wakker, 2008. "Explaining the characteristics of the power (CRRA) utility family," Health Economics, John Wiley & Sons, Ltd., vol. 17(12), pages 1329-1344.
    30. Robert B. Barsky & F. Thomas Juster & Miles S. Kimball & Matthew D. Shapiro, 1997. "Preference Parameters and Behavioral Heterogeneity: An Experimental Approach in the Health and Retirement Study," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 537-579.
    31. Binswanger, Hans P, 1981. "Attitudes toward Risk: Theoretical Implications of an Experiment in Rural India," Economic Journal, Royal Economic Society, vol. 91(364), pages 867-890, December.
    32. Deck, Cary & Lee, Jungmin & Reyes, Javier A. & Rosen, Christopher C., 2013. "A failed attempt to explain within subject variation in risk taking behavior using domain specific risk attitudes," Journal of Economic Behavior & Organization, Elsevier, vol. 87(C), pages 1-24.
    33. Arnaud Reynaud & Stéphane Couture, 2012. "Stability of risk preference measures: results from a field experiment on French farmers," Theory and Decision, Springer, vol. 73(2), pages 203-221, August.
    34. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    35. Sen, Amartya, 1993. "Internal Consistency of Choice," Econometrica, Econometric Society, vol. 61(3), pages 495-521, May.
    36. Tversky, Amos & Slovic, Paul & Kahneman, Daniel, 1990. "The Causes of Preference Reversal," American Economic Review, American Economic Association, vol. 80(1), pages 204-217, March.
    37. Charles A. Holt & Susan K. Laury, 2005. "Risk Aversion and Incentive Effects: New Data without Order Effects," American Economic Review, American Economic Association, vol. 95(3), pages 902-912, June.
    38. Holzmeister, Felix, 2017. "oTree: Ready-made apps for risk preference elicitation methods," Journal of Behavioral and Experimental Finance, Elsevier, vol. 16(C), pages 33-38.
    39. Peter P. Wakker, 2008. "Explaining the characteristics of the power (CRRA) utility family," Health Economics, John Wiley & Sons, Ltd., vol. 17(12), pages 1329-1344, December.
    40. Isaac, R Mark & James, Duncan, 2000. "Just Who Are You Calling Risk Averse?," Journal of Risk and Uncertainty, Springer, vol. 20(2), pages 177-187, March.
    41. Meraner, Manuela & Musshoff, Oliver & Finger, Robert, 2018. "Using involvement to reduce inconsistencies in risk preference elicitation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 73(C), pages 22-33.
    42. Sugden, Robert, 1991. "Rational Choice: A Survey of Contributions from Economics and Philosophy," Economic Journal, Royal Economic Society, vol. 101(407), pages 751-785, July.
    43. Peter Wakker & Daniel Deneffe, 1996. "Eliciting von Neumann-Morgenstern Utilities When Probabilities Are Distorted or Unknown," Management Science, INFORMS, vol. 42(8), pages 1131-1150, August.
    44. Mohammed Abdellaoui & Ahmed Driouchi & Olivier L’Haridon, 2011. "Risk aversion elicitation: reconciling tractability and bias minimization," Theory and Decision, Springer, vol. 71(1), pages 63-80, July.
    45. Catherine C. Eckel & Philip J. Grossman, 2008. "Forecasting Risk Attitudes: An Experimental Study Using Actual and Forecast Gamble Choices," Monash Economics Working Papers archive-01, Monash University, Department of Economics.
    46. David Bruner, 2009. "Changing the probability versus changing the reward," Experimental Economics, Springer;Economic Science Association, vol. 12(4), pages 367-385, December.
    47. Slovic, Paul, 1972. "Psychological Study of Human Judgment: Implications for Investment Decision-Making," Journal of Finance, American Finance Association, vol. 27(4), pages 779-799, September.
    48. Bock, Olaf & Baetge, Ingmar & Nicklisch, Andreas, 2014. "hroot: Hamburg Registration and Organization Online Tool," European Economic Review, Elsevier, vol. 71(C), pages 117-120.
    49. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    50. Steffen Andersen & Glenn Harrison & Morten Lau & E. Rutström, 2009. "Elicitation using multiple price list formats," Experimental Economics, Springer;Economic Science Association, vol. 12(3), pages 365-366, September.
    51. Tomomi Tanaka & Colin F. Camerer & Quang Nguyen, 2010. "Risk and Time Preferences: Linking Experimental and Household Survey Data from Vietnam," American Economic Review, American Economic Association, vol. 100(1), pages 557-571, March.
    52. Jonathan P. Beauchamp & David Cesarini & Magnus Johannesson, 2017. "The psychometric and empirical properties of measures of risk preferences," Journal of Risk and Uncertainty, Springer, vol. 54(3), pages 203-237, June.
    53. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    54. Rui Mata & Renato Frey & David Richter & Jürgen Schupp & Ralph Hertwig, 2018. "Risk Preference: A View from Psychology," Journal of Economic Perspectives, American Economic Association, vol. 32(2), pages 155-172, Spring.
    55. Tversky, Amos & Thaler, Richard H, 1990. "Anomalies: Preference Reversals," Journal of Economic Perspectives, American Economic Association, vol. 4(2), pages 201-211, Spring.
    56. Charness, Gary & Gneezy, Uri & Imas, Alex, 2013. "Experimental methods: Eliciting risk preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 87(C), pages 43-51.
    57. Enrica Carbone & John D. Hey, 1995. "A Comparison of the Estimates of Expected Utility and Non-Expected-Utility Preference Functionals," The Geneva Risk and Insurance Review, Palgrave Macmillan;International Association for the Study of Insurance Economics (The Geneva Association), vol. 20(1), pages 111-133, June.
    58. Chris Starmer, 2000. "Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 332-382, June.
    59. Chetan Dave & Catherine Eckel & Cathleen Johnson & Christian Rojas, 2010. "Eliciting risk preferences: When is simple better?," Journal of Risk and Uncertainty, Springer, vol. 41(3), pages 219-243, December.
    60. Ola Andersson & Håkan J. Holm & Jean-Robert Tyran & Erik Wengström, 2020. "Robust inference in risk elicitation tasks," Journal of Risk and Uncertainty, Springer, vol. 61(3), pages 195-209, December.
    61. Graham Loomes & Ganna Pogrebna, 2014. "Measuring Individual Risk Attitudes when Preferences are Imprecise," Economic Journal, Royal Economic Society, vol. 0(576), pages 569-593, May.
    62. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    63. Sarah Jacobson & Ragan Petrie, 2009. "Learning from mistakes: What do inconsistent choices over risk tell us?," Journal of Risk and Uncertainty, Springer, vol. 38(2), pages 143-158, April.
    64. Camerer, Colin F & Ho, Teck-Hua, 1994. "Violations of the Betweenness Axiom and Nonlinearity in Probability," Journal of Risk and Uncertainty, Springer, vol. 8(2), pages 167-196, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Felix Holzmeister & Matthias Stefan, 2019. "The risk elicitation puzzle revisited: Across-methods (in)consistency?," Working Papers 2019-19, Faculty of Economics and Statistics, Universität Innsbruck.
    2. Felix Holzmeister & Matthias Stefan, 2021. "The risk elicitation puzzle revisited: Across-methods (in)consistency?," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 593-616, June.
    3. Tamás Csermely & Alexander Rabas, 2016. "How to reveal people’s preferences: Comparing time consistency and predictive power of multiple price list risk elicitation methods," Journal of Risk and Uncertainty, Springer, vol. 53(2), pages 107-136, December.
    4. Galizzi, Matteo M. & Machado, Sara R. & Miniaci, Raffaele, 2016. "Temporal stability, cross-validity, and external validity of risk preferences measures: experimental evidence from a UK representative sample," LSE Research Online Documents on Economics 67554, London School of Economics and Political Science, LSE Library.
    5. Paolo Crosetto & Antonio Filippin, 2016. "A theoretical and experimental appraisal of four risk elicitation methods," Experimental Economics, Springer;Economic Science Association, vol. 19(3), pages 613-641, September.
    6. Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
    7. Menkhoff, Lukas & Sakha, Sahra, 2017. "Estimating risky behavior with multiple-item risk measures," Journal of Economic Psychology, Elsevier, vol. 59(C), pages 59-86.
    8. Holzmeister, Felix, 2017. "oTree: Ready-made apps for risk preference elicitation methods," Journal of Behavioral and Experimental Finance, Elsevier, vol. 16(C), pages 33-38.
    9. Arianna Galliera & E. Elisabet Rutström, 2021. "Crowded out: Heterogeneity in risk attitudes among poor households in the US," Journal of Risk and Uncertainty, Springer, vol. 63(2), pages 103-132, October.
    10. Bauermeister, Golo-Friedrich & Mußhoff, Oliver, 2016. "Risk Attitude And Inconsistencies - Does The Choice Of Display Format And Risk Elicitation Method Influence The Outcomes?," 56th Annual Conference, Bonn, Germany, September 28-30, 2016 244764, German Association of Agricultural Economists (GEWISOLA).
    11. Golo-Friedrich Bauermeister & Daniel Hermann & Oliver Musshoff, 2018. "Consistency of determined risk attitudes and probability weightings across different elicitation methods," Theory and Decision, Springer, vol. 84(4), pages 627-644, June.
    12. Ranganathan, Kavitha & Lejarraga, Tomás, 2021. "Elicitation of risk preferences through satisficing," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    13. Galarza, Francisco, 2009. "Choices under Risk in Rural Peru," MPRA Paper 17708, University Library of Munich, Germany.
    14. Charness, Gary & Gneezy, Uri & Imas, Alex, 2013. "Experimental methods: Eliciting risk preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 87(C), pages 43-51.
    15. Menkhoff, Lukas & Sakha, Sahra, 2014. "Multiple-item risk measures," Kiel Working Papers 1980, Kiel Institute for the World Economy (IfW Kiel).
    16. Michele Garagnani, 2023. "The predictive power of risk elicitation tasks," Journal of Risk and Uncertainty, Springer, vol. 67(2), pages 165-192, October.
    17. Bauermeister, Golo & Musshoff, Oliver, 2016. "Risk Aversion and Inconsistencies - Does the Choice of Risk Elicitation Method and Display Format Influence the Outcomes?," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 235348, Agricultural and Applied Economics Association.
    18. Lönnqvist, Jan-Erik & Verkasalo, Markku & Walkowitz, Gari & Wichardt, Philipp C., 2015. "Measuring individual risk attitudes in the lab: Task or ask? An empirical comparison," Journal of Economic Behavior & Organization, Elsevier, vol. 119(C), pages 254-266.
    19. Filippin, Antonio & Crosetto, Paolo, 2014. "A Reconsideration of Gender Differences in Risk Attitudes," IZA Discussion Papers 8184, Institute of Labor Economics (IZA).
    20. Paolo Crosetto & Antonio Filippin, 2013. "A Theoretical and Experimental Appraisal of Five Risk Elicitation Methods," SOEPpapers on Multidisciplinary Panel Data Research 547, DIW Berlin, The German Socio-Economic Panel (SOEP).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:pj9u2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.