IDEAS home Printed from https://ideas.repec.org/a/kap/jrisku/v39y2009i3p213-235.html
   My bibliography  Save this article

Noise and bias in eliciting preferences

Author

Listed:
  • John Hey
  • Andrea Morone
  • Ulrich Schmidt

Abstract

In the context of eliciting preferences for decision making under risk, we ask the question: "which might be the 'best' method for eliciting such preferences?". It is well known that different methods differ in terms of the bias in the elicitation; it is rather less well-known that different methods differ in terms of their noisiness. The optimal trade-off depends upon the relative magnitudes of these two effects. We examine four different elicitation mechanisms (pairwise choice, willingness-to-pay, willingness-to-accept, and certainty equivalents) and estimate both effects. Our results suggest that economists might be better advised to use what appears to be a relatively inefficient elicitation technique (i.e. pairwise choice) in order to avoid the bias in better-known and more widely-used techniques.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • John Hey & Andrea Morone & Ulrich Schmidt, 2009. "Noise and bias in eliciting preferences," Journal of Risk and Uncertainty, Springer, vol. 39(3), pages 213-235, December.
  • Handle: RePEc:kap:jrisku:v:39:y:2009:i:3:p:213-235
    DOI: 10.1007/s11166-009-9081-1
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11166-009-9081-1
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s11166-009-9081-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Loomes, Graham & Moffatt, Peter G & Sugden, Robert, 2002. "A Microeconometric Test of Alternative Stochastic Theories of Risky Choice," Journal of Risk and Uncertainty, Springer, vol. 24(2), pages 103-130, March.
    2. John H. Kagel & Colin M. Campbell & Dan Levin, 1999. "The Winner's Curse and Public Information in Common Value Auctions: Reply," American Economic Review, American Economic Association, vol. 89(1), pages 325-334, March.
    3. Starmer, Chris & Sugden, Robert, 1989. "Probability and Juxtaposition Effects: An Experimental Investigation of the Common Ratio Effect," Journal of Risk and Uncertainty, Springer, vol. 2(2), pages 159-178, June.
    4. Jack L. Knetsch & J. A. Sinden, 1984. "Willingness to Pay and Compensation Demanded: Experimental Evidence of an Unexpected Disparity in Measures of Value," The Quarterly Journal of Economics, Oxford University Press, vol. 99(3), pages 507-521.
    5. Beattie, Jane & Loomes, Graham, 1997. "The Impact of Incentives upon Risky Choice Experiments," Journal of Risk and Uncertainty, Springer, vol. 14(2), pages 155-168, March.
    6. Dyer, Douglas & Kagel, John H & Levin, Dan, 1989. "A Comparison of Naive and Experienced Bidders in Common Value Offer Auctions: A Laboratory Analysis," Economic Journal, Royal Economic Society, vol. 99(394), pages 108-115, March.
    7. Ulrich Schmidt & John D. Hey, 2018. "Are Preference Reversals Errors? An Experimental Investigation," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 15, pages 353-364, World Scientific Publishing Co. Pte. Ltd..
    8. Duncan James, 2007. "Stability of risk preference parameter estimates within the Becker-DeGroot-Marschak procedure," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 123-141, June.
    9. Michael Birnbaum & Ulrich Schmidt, 2008. "An experimental investigation of violations of transitivity in choice under uncertainty," Journal of Risk and Uncertainty, Springer, vol. 37(1), pages 77-91, August.
    10. Glenn Harrison & E. Rutström, 2009. "Expected utility theory and prospect theory: one wedding and a decent funeral," Experimental Economics, Springer;Economic Science Association, vol. 12(2), pages 133-158, June.
    11. Quiggin, John, 1982. "A theory of anticipated utility," Journal of Economic Behavior & Organization, Elsevier, vol. 3(4), pages 323-343, December.
    12. Isaac, R. Mark & Walker, James M., 1985. "Information and conspiracy in sealed bid auctions," Journal of Economic Behavior & Organization, Elsevier, vol. 6(2), pages 139-159, June.
    13. Enrica Carbone & John D. Hey, 2018. "Which Error Story is Best?," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 16, pages 365-380, World Scientific Publishing Co. Pte. Ltd..
    14. John D. Hey & Chris Orme, 2018. "Investigating Generalizations Of Expected Utility Theory Using Experimental Data," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 3, pages 63-98, World Scientific Publishing Co. Pte. Ltd..
    15. Lisa Anderson & Jennifer Mellor, 2009. "Are risk preferences stable? Comparing an experimental measure with a validated survey-based measure," Journal of Risk and Uncertainty, Springer, vol. 39(2), pages 137-160, October.
    16. Coppinger, Vicki M & Smith, Vernon L & Titus, Jon A, 1980. "Incentives and Behavior in English, Dutch and Sealed-Bid Auctions," Economic Inquiry, Western Economic Association International, vol. 18(1), pages 1-22, January.
    17. Harless, David W & Camerer, Colin F, 1994. "The Predictive Utility of Generalized Expected Utility Theories," Econometrica, Econometric Society, vol. 62(6), pages 1251-1289, November.
    18. John Hey & Jinkwon Lee, 2005. "Do Subjects Separate (or Are They Sophisticated)?," Experimental Economics, Springer;Economic Science Association, vol. 8(3), pages 233-265, September.
    19. Don L. Coursey & John L. Hovis & William D. Schulze, 1987. "The Disparity Between Willingness to Accept and Willingness to Pay Measures of Value," The Quarterly Journal of Economics, Oxford University Press, vol. 102(3), pages 679-690.
    20. Samuelson, William & Zeckhauser, Richard, 1988. "Status Quo Bias in Decision Making," Journal of Risk and Uncertainty, Springer, vol. 1(1), pages 7-59, March.
    21. Camerer, Colin F, 1989. "An Experimental Test of Several Generalized Utility Theories," Journal of Risk and Uncertainty, Springer, vol. 2(1), pages 61-104, April.
    22. Robin Cubitt & Chris Starmer & Robert Sugden, 1998. "On the Validity of the Random Lottery Incentive System," Experimental Economics, Springer;Economic Science Association, vol. 1(2), pages 115-131, September.
    23. Jack L. Knetsch & J. A. Sinden, 1987. "The Persistence of Evaluation Disparities," The Quarterly Journal of Economics, Oxford University Press, vol. 102(3), pages 691-695.
    24. Ulrich Schmidt & Stefan Traub, 2009. "An Experimental Investigation of the Disparity Between WTA and WTP for Lotteries," Theory and Decision, Springer, vol. 66(3), pages 229-262, March.
    25. David Buschena & David Zilberman, 2008. "Generalized expected utility, heteroscedastic error, and path dependence in risky choice," Journal of Risk and Uncertainty, Springer, vol. 36(2), pages 201-201, April.
    26. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    27. Carbone, Enrica, 1997. "Investigation of stochastic preference theory using experimental data," Economics Letters, Elsevier, vol. 57(3), pages 305-311, December.
    28. Blavatskyy, Pavlo R., 2008. "Stochastic utility theorem," Journal of Mathematical Economics, Elsevier, vol. 44(11), pages 1049-1056, December.
    29. Lind, Barry & Plott, Charles R, 1991. "The Winner's Curse: Experiments with Buyers and with Sellers," American Economic Review, American Economic Association, vol. 81(1), pages 335-346, March.
    30. Susan K. Laury, 2006. "Pay One or Pay All: Random Selection of One Choice for Payment," Experimental Economics Center Working Paper Series 2006-24, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    31. Wu, George, 1994. "An Empirical Test of Ordinal Independence," Journal of Risk and Uncertainty, Springer, vol. 9(1), pages 39-60, July.
    32. Wilcox, Nathaniel T., 2011. "'Stochastically more risk averse:' A contextual theory of stochastic discrete choice under risk," Journal of Econometrics, Elsevier, vol. 162(1), pages 89-104, May.
    33. Pavlo Blavatskyy, 2007. "Stochastic expected utility theory," Journal of Risk and Uncertainty, Springer, vol. 34(3), pages 259-286, June.
    34. Ballinger, T Parker & Wilcox, Nathaniel T, 1997. "Decisions, Error and Heterogeneity," Economic Journal, Royal Economic Society, vol. 107(443), pages 1090-1105, July.
    35. Chris Starmer, 2000. "Developments in Non-expected Utility Theory: The Hunt for a Descriptive Theory of Choice under Risk," Journal of Economic Literature, American Economic Association, vol. 38(2), pages 332-382, June.
    36. Pavlo R. Blavatskyy, "undated". "A Stochastic Expected Utility Theory," IEW - Working Papers 231, Institute for Empirical Research in Economics - University of Zurich.
    37. Kagel, John H. & Levin, Dan, 1986. "The Winner's Curse and Public Information in Common Value Auctions," American Economic Review, American Economic Association, vol. 76(5), pages 894-920, December.
    38. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    39. Isaac, R Mark & James, Duncan, 2000. "Just Who Are You Calling Risk Averse?," Journal of Risk and Uncertainty, Springer, vol. 20(2), pages 177-187, March.
    40. Harrison, Glenn W, 1990. "Risk Attitudes in First-Price Auction Experiments: A Bayesian Analysis," The Review of Economics and Statistics, MIT Press, vol. 72(3), pages 541-546, August.
    41. Michael Birnbaum & Ulrich Schmidt, 2010. "Testing transitivity in choice under risk," Theory and Decision, Springer, vol. 69(4), pages 599-614, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Morone, Andrea, 2010. "On price data elicitation: A laboratory investigation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(5), pages 540-545, October.
    2. Michael H. Birnbaum & Ulrich Schmidt & Miriam D. Schneider, 2017. "Testing independence conditions in the presence of errors and splitting effects," Journal of Risk and Uncertainty, Springer, vol. 54(1), pages 61-85, February.
    3. Henry Stott, 2006. "Cumulative prospect theory's functional menagerie," Journal of Risk and Uncertainty, Springer, vol. 32(2), pages 101-130, March.
    4. David M. Bruner, 2017. "Does decision error decrease with risk aversion?," Experimental Economics, Springer;Economic Science Association, vol. 20(1), pages 259-273, March.
    5. Andreas C Drichoutis & Jayson L Lusk, 2014. "Judging Statistical Models of Individual Decision Making under Risk Using In- and Out-of-Sample Criteria," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-13, July.
    6. Wilcox, Nathaniel T., 2011. "'Stochastically more risk averse:' A contextual theory of stochastic discrete choice under risk," Journal of Econometrics, Elsevier, vol. 162(1), pages 89-104, May.
    7. Pavlo Blavatskyy, 2014. "Stronger utility," Theory and Decision, Springer, vol. 76(2), pages 265-286, February.
    8. Aurora García-Gallego & Nikolaos Georgantzís & Daniel Navarro-Martínez & Gerardo Sabater-Grande, 2011. "The stochastic component in choice and regression to the mean," Theory and Decision, Springer, vol. 71(2), pages 251-267, August.
    9. Anna Conte & John D. Hey & Peter G. Moffatt, 2018. "Mixture models of choice under risk," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 1, pages 3-12, World Scientific Publishing Co. Pte. Ltd..
    10. Pavlo Blavatskyy, 2012. "Probabilistic choice and stochastic dominance," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 50(1), pages 59-83, May.
    11. Pavlo Blavatskyy, 2007. "Stochastic expected utility theory," Journal of Risk and Uncertainty, Springer, vol. 34(3), pages 259-286, June.
    12. Pavlo R. Blavatskyy & Ganna Pogrebna, 2010. "Models of stochastic choice and decision theories: why both are important for analyzing decisions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(6), pages 963-986.
    13. Nathaniel T. Wilcox, 2015. "Error and Generalization in Discrete Choice Under Risk," Working Papers 15-11, Chapman University, Economic Science Institute.
    14. Glenn Harrison & J. Swarthout, 2014. "Experimental payment protocols and the Bipolar Behaviorist," Theory and Decision, Springer, vol. 77(3), pages 423-438, October.
    15. James Cox & Vjollca Sadiraj & Ulrich Schmidt, 2015. "Paradoxes and mechanisms for choice under risk," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 215-250, June.
    16. Pavlo Blavatskyy, 2018. "A second-generation disappointment aversion theory of decision making under risk," Theory and Decision, Springer, vol. 84(1), pages 29-60, January.
    17. Glenn W. Harrison & J. Todd Swarthout, 2016. "Cumulative Prospect Theory in the Laboratory: A Reconsideration," Experimental Economics Center Working Paper Series 2016-04, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    18. Jakusch, Sven Thorsten & Meyer, Steffen & Hackethal, Andreas, 2019. "Taming models of prospect theory in the wild? Estimation of Vlcek and Hens (2011)," SAFE Working Paper Series 146, Leibniz Institute for Financial Research SAFE.
    19. John D. Hey, 2018. "Why We Should Not Be Silent About Noise," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 13, pages 309-329, World Scientific Publishing Co. Pte. Ltd..
    20. Egil Matsen & Bjarne Strøm, 2006. "Joker: Choice in a simple game with large stakes," Working Paper Series 8307, Department of Economics, Norwegian University of Science and Technology.

    More about this item

    Keywords

    Pairwise choice; Willingness-to-pay; Willingness-to-accept; BDM mechanism; Errors; Noise; Biases; JEL classifications ; C91; C81;
    All these keywords.

    JEL classification:

    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jrisku:v:39:y:2009:i:3:p:213-235. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.