IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v134y2017icp430-448.html
   My bibliography  Save this article

Scoring rules for subjective probability distributions

Author

Listed:
  • Harrison, Glenn W.
  • Martínez-Correa, Jimmy
  • Swarthout, J. Todd
  • Ulm, Eric R.

Abstract

Subjective beliefs are elicited routinely in economics experiments. However, such elicitation often suffers from two possible disadvantages. First, beliefs are recovered in the form of a summary statistic, usually the mean, of the underlying latent distribution. Second, recovered beliefs are biased significantly due to risk aversion. We characterize an approach for eliciting the entire subjective belief distribution that is minimally biased due to risk aversion. We offer simulated examples to demonstrate the intuition of our approach. We also provide theory to formally characterize our framework. And we provide experimental evidence which corroborates our theoretical results. We conclude that for empirically plausible levels of risk aversion, one can reliably elicit most important features of the latent subjective belief distribution without undertaking calibration for risk attitudes providing one is willing to assume Subjective Expected Utility.

Suggested Citation

  • Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd & Ulm, Eric R., 2017. "Scoring rules for subjective probability distributions," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 430-448.
  • Handle: RePEc:eee:jeborg:v:134:y:2017:i:c:p:430-448
    DOI: 10.1016/j.jebo.2016.12.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268116302839
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jebo.2016.12.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    2. Constantinos Antoniou & Glenn Harrison & Morten Lau & Daniel Read, 2015. "Subjective Bayesian beliefs," Journal of Risk and Uncertainty, Springer, vol. 50(1), pages 35-54, February.
    3. Kenneth C. Lichtendahl, Jr. & Robert L. Winkler, 2007. "Probability Elicitation, Scoring Rules, and Competition Among Forecasters," Management Science, INFORMS, vol. 53(11), pages 1745-1755, November.
    4. James E. Matheson & Robert L. Winkler, 1976. "Scoring Rules for Continuous Probability Distributions," Management Science, INFORMS, vol. 22(10), pages 1087-1096, June.
    5. Caplin, Andrew & Schotter, Andrew, 2008. "The Foundations of Positive and Normative Economics: A Handbook," OUP Catalogue, Oxford University Press, number 9780195328318.
    6. Holt, Charles A. & Smith, Angela M., 2009. "An update on Bayesian updating," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 125-134, February.
    7. Glenn W. Harrison & Jia Min Ng, 2016. "Evaluating The Expected Welfare Gain From Insurance," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 83(1), pages 91-120, January.
    8. Edi Karni, 2009. "A Mechanism for Eliciting Probabilities," Econometrica, Econometric Society, vol. 77(2), pages 603-606, March.
    9. Charles A. Holt & Susan K. Laury, 2005. "Risk Aversion and Incentive Effects: New Data without Order Effects," American Economic Review, American Economic Association, vol. 95(3), pages 902-912, June.
    10. Robert F. Nau, 2006. "Uncertainty Aversion with Second-Order Utilities and Probabilities," Management Science, INFORMS, vol. 52(1), pages 136-145, January.
    11. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    12. Grether, David M., 1992. "Testing bayes rule and the representativeness heuristic: Some experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 17(1), pages 31-57, January.
    13. Merkle, Christoph & Weber, Martin, 2011. "True overconfidence: The inability of rational information processing to account for apparent overconfidence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 116(2), pages 262-271.
    14. Costa-Gomes, Miguel A. & Huck, Steffen & Weizsäcker, Georg, 2014. "Beliefs and actions in the trust game: Creating instrumental variables to estimate the causal effect," Games and Economic Behavior, Elsevier, vol. 88(C), pages 298-309.
    15. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2014. "Eliciting subjective probabilities with binary lotteries," Journal of Economic Behavior & Organization, Elsevier, vol. 101(C), pages 128-140.
    16. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2015. "Reduction of compound lotteries with objective probabilities: Theory and evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 119(C), pages 32-55.
    17. Delavande, Adeline & Giné, Xavier & McKenzie, David, 2011. "Measuring subjective expectations in developing countries: A critical review and new evidence," Journal of Development Economics, Elsevier, vol. 94(2), pages 151-163, March.
    18. Offerman, Theo & Sonnemans, Joep & Schram, Arthur, 1996. "Value Orientations, Expectations and Voluntary Contributions in Public Goods," Economic Journal, Royal Economic Society, vol. 106(437), pages 817-845, July.
    19. Fiore, Stephen M. & Harrison, Glenn W. & Hughes, Charles E. & Rutstrm, E. Elisabet, 2009. "Virtual experiments and environmental policy," Journal of Environmental Economics and Management, Elsevier, vol. 57(1), pages 65-86, January.
    20. Glenn Harrison & J. Swarthout, 2014. "Experimental payment protocols and the Bipolar Behaviorist," Theory and Decision, Springer, vol. 77(3), pages 423-438, October.
    21. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    22. Jeff Dominitz & Charles F. Manski, 2011. "Measuring and interpreting expectations of equity returns," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 352-370, April.
    23. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    24. Glenn W. Harrison & J. Todd Swarthout, 2012. "The Independence Axiom and the Bipolar Behaviorist," Experimental Economics Center Working Paper Series 2012-01, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    25. Antoniou, Constantinos & Harrison, Glenn W. & Lau, Morten I. & Read, Daniel, 2017. "Information Characteristics and Errors in Expectations: Experimental Evidence," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 52(2), pages 737-750, April.
    26. Ergin, Haluk & Gul, Faruk, 2009. "A theory of subjective compound lotteries," Journal of Economic Theory, Elsevier, vol. 144(3), pages 899-929, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:exc:wpaper:2013-05 is not listed on IDEAS
    2. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2014. "Eliciting subjective probabilities with binary lotteries," Journal of Economic Behavior & Organization, Elsevier, vol. 101(C), pages 128-140.
    3. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    4. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    5. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    6. Amalia Di Girolamo & Glenn W. Harrison & Morten I. Lau & J. Todd Swarthout, 2013. "Characterizing Financial and Statistical Literacy," Experimental Economics Center Working Paper Series 2013-04, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    7. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    8. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    9. Di Girolamo, Amalia & Harrison, Glenn W. & Lau, Morten I. & Swarthout, J. Todd, 2015. "Subjective belief distributions and the characterization of economic literacy," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 59(C), pages 1-12.
    10. Constantinos Antoniou & Glenn Harrison & Morten Lau & Daniel Read, 2015. "Subjective Bayesian beliefs," Journal of Risk and Uncertainty, Springer, vol. 50(1), pages 35-54, February.
    11. Crosetto, Paolo & Filippin, Antonio & Katuščák, Peter & Smith, John, 2020. "Central tendency bias in belief elicitation," Journal of Economic Psychology, Elsevier, vol. 78(C).
    12. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    13. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2010. "Subjective beliefs formation and elicitation rules: experimental evidence," Documents de travail du Centre d'Economie de la Sorbonne 10088, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    14. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2016. "In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments," Theory and Decision, Springer, vol. 80(3), pages 363-387, March.
    15. Li Hao & Daniel Houser, 2012. "Belief elicitation in the presence of naïve respondents: An experimental study," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 161-180, April.
    16. Ambuehl, Sandro & Li, Shengwu, 2018. "Belief updating and the demand for information," Games and Economic Behavior, Elsevier, vol. 109(C), pages 21-39.
    17. Cheung, Stephen L. & Johnstone, Lachlan, 2017. "True Overconfidence, Revealed through Actions: An Experiment," IZA Discussion Papers 10545, Institute of Labor Economics (IZA).
    18. Alexander Coutts, 2019. "Good news and bad news are still news: experimental evidence on belief updating," Experimental Economics, Springer;Economic Science Association, vol. 22(2), pages 369-395, June.
    19. Loic Berger & Valentina Bosetti, 2016. "Ellsberg Re-revisited: An Experiment Disentangling Model Uncertainty and Risk Aversion," Working Papers 2016.37, Fondazione Eni Enrico Mattei.
    20. Ingrid Burfurd & Tom Wilkening, 2022. "Cognitive heterogeneity and complex belief elicitation," Experimental Economics, Springer;Economic Science Association, vol. 25(2), pages 557-592, April.
    21. Simone Cerroni, 2020. "Eliciting farmers’ subjective probabilities, risk, and uncertainty preferences using contextualized field experiments," Agricultural Economics, International Association of Agricultural Economists, vol. 51(5), pages 707-724, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:134:y:2017:i:c:p:430-448. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.