IDEAS home Printed from https://ideas.repec.org/p/unm/umagsb/2018023.html
   My bibliography  Save this paper

Robust scoring rules

Author

Listed:
  • Tsakas, Elias

    (Microeconomics & Public Economics, RS: GSBE ETBC, RS: GSBE Theme Conflict & Cooperation, RS: GSBE Theme Human Decisions and Policy Design)

Abstract

We study elicitation of latent (prior) beliefs when the agent can acquire information via a costly attention strategy. We introduce a mechanism that simultaneously makes it strictly dominant to (a) not acquire any information, and (b) report truthfully. We call such a mechanism a robust scoring rule. Robust scoring rules are important for different reasons. Theoretically, they are crucial both for establishing that decision-theoretic models under uncertainty are testable. From an applied point of view, they are needed for eliciting unbiased estimates of population beliefs. We prove that a robust scoring rule exists under mild axioms on the attention costs. These axioms are shown to characterize the class of posterior-separable cost functions. Our existence proof is constructive, thus identifying an entire class of robust scoring rules. Subsequently, we show that we can arbitrarily approximate the agent's prior beliefs with a quadratic scoring rule. The same holds true for a discrete scoring rule. Finally, we show that the prior beliefs can be approximated, even when we are uncertain about the exact specification of the agent's attention costs.

Suggested Citation

  • Tsakas, Elias, 2018. "Robust scoring rules," Research Memorandum 023, Maastricht University, Graduate School of Business and Economics (GSBE).
  • Handle: RePEc:unm:umagsb:2018023
    DOI: 10.26481/umagsb.2018023
    as

    Download full text from publisher

    File URL: https://cris.maastrichtuniversity.nl/ws/files/29754027/RM18023.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.26481/umagsb.2018023?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    2. Reinhard Selten & Abdolkarim Sadrieh & Klaus Abbink, 1999. "Money Does Not Induce Risk Neutral Behavior, but Binary Lotteries Do even Worse," Theory and Decision, Springer, vol. 46(3), pages 213-252, June.
    3. Emir Kamenica & Matthew Gentzkow, 2011. "Bayesian Persuasion," American Economic Review, American Economic Association, vol. 101(6), pages 2590-2615, October.
    4. Bartosz Mackowiak & Mirko Wiederholt, 2009. "Optimal Sticky Prices under Rational Inattention," American Economic Review, American Economic Association, vol. 99(3), pages 769-803, June.
    5. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    6. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    7. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    8. Larbi Alaoui & Antonio Penta, 2018. "Cost-benefit analysis in reasoning," Economics Working Papers 1621, Department of Economics and Business, Universitat Pompeu Fabra.
    9. Filip Matějka & Guido Tabellini, 2021. "Electoral Competition with Rationally Inattentive Voters," Journal of the European Economic Association, European Economic Association, vol. 19(3), pages 1899-1935.
    10. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    11. Vojtěch Bartoš & Michal Bauer & Julie Chytilová & Filip Matějka, 2016. "Attention Discrimination: Theory and Field Experiments with Monitoring Information Acquisition," American Economic Review, American Economic Association, vol. 106(6), pages 1437-1475, June.
    12. William Grisley & Earl D. Kellogg, 1983. "Farmers' Subjective Probabilities in Northern Thailand: An Elicitation Analysis," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 65(1), pages 74-82.
    13. Maccheroni, Fabio & Marinacci, Massimo & Rustichini, Aldo, 2006. "Dynamic variational preferences," Journal of Economic Theory, Elsevier, vol. 128(1), pages 4-44, May.
    14. Andrew Caplin & Mark Dean & John Leahy, 2022. "Rationally Inattentive Behavior: Characterizing and Generalizing Shannon Entropy," Journal of Political Economy, University of Chicago Press, vol. 130(6), pages 1676-1715.
    15. Chambers, Christopher P., 2008. "Proper scoring rules for general decision models," Games and Economic Behavior, Elsevier, vol. 63(1), pages 32-40, May.
    16. Palfrey, Thomas R. & Wang, Stephanie W., 2009. "On eliciting beliefs in strategic games," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 98-109, August.
    17. Filip Matějka, 2016. "Rationally Inattentive Seller: Sales and Discrete Pricing," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(3), pages 1125-1155.
    18. Andrew Caplin, 2016. "Measuring and Modeling Attention," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 379-403, October.
    19. Jaffray, Jean-Yves & Karni, Edi, 1999. "Elicitation of Subjective Probabilities When the Initial Endowment is Unobservable," Journal of Risk and Uncertainty, Springer, vol. 18(1), pages 5-20, April.
    20. Andrew Caplin & Mark Dean, 2015. "Revealed Preference, Rational Inattention, and Costly Information Acquisition," American Economic Review, American Economic Association, vol. 105(7), pages 2183-2203, July.
    21. Bergemann, Dirk & Morris, Stephen & Takahashi, Satoru, 2017. "Interdependent preferences and strategic distinguishability," Journal of Economic Theory, Elsevier, vol. 168(C), pages 329-371.
    22. de Oliveira, Henrique & Denti, Tommaso & Mihm, Maximilian & Ozbek, Kemal, 2017. "Rationally inattentive preferences and hidden information costs," Theoretical Economics, Econometric Society, vol. 12(2), May.
    23. Michael Ostrovsky, 2012. "Information Aggregation in Dynamic Markets With Strategic Traders," Econometrica, Econometric Society, vol. 80(6), pages 2595-2647, November.
    24. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    25. Michael D. Hurd, 2009. "Subjective Probabilities in Household Surveys," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 543-564, May.
    26. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    27. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    28. Thomson, William, 1979. "Eliciting production possibilities from a well-informed manager," Journal of Economic Theory, Elsevier, vol. 20(3), pages 360-380, June.
    29. Chambers,Christopher P. & Echenique,Federico, 2016. "Revealed Preference Theory," Cambridge Books, Cambridge University Press, number 9781107087804.
    30. Theo Offerman & Joep Sonnemans & Gijs Van De Kuilen & Peter P. Wakker, 2009. "A Truth Serum for Non-Bayesians: Correcting Proper Scoring Rules for Risk Attitudes ," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(4), pages 1461-1489.
    31. Uri Gneezy, 2005. "Deception: The Role of Consequences," American Economic Review, American Economic Association, vol. 95(1), pages 384-394, March.
    32. Ellis, Andrew, 2018. "Foundations for optimal inattention," Journal of Economic Theory, Elsevier, vol. 173(C), pages 56-94.
    33. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    34. Robert Clemen, 2002. "Incentive contrats and strictly proper scoring rules," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 11(1), pages 167-189, June.
    35. Harrison, Glenn W. & Martínez-Correa, Jimmy & Swarthout, J. Todd, 2013. "Inducing risk neutral preferences with binary lotteries: A reconsideration," Journal of Economic Behavior & Organization, Elsevier, vol. 94(C), pages 145-159.
    36. Robin Hanson, 2003. "Combinatorial Information Market Design," Information Systems Frontiers, Springer, vol. 5(1), pages 107-119, January.
    37. Christopher P. Chambers & Nicolas S. Lambert, 2021. "Dynamic Belief Elicitation," Econometrica, Econometric Society, vol. 89(1), pages 375-414, January.
    38. Hebert, Benjamin & Woodford, Michael, 2017. "Rational Inattention with Sequential Information Sampling," Research Papers repec:ecl:stabus:3457, Stanford University, Graduate School of Business.
    39. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    40. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    41. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-1339, November.
    42. Karni, Edi, 1999. "Elicitation of Subjective Probabilities When Preferences Are State-Dependent," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 479-486, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chambers, Christopher P. & Healy, Paul J. & Lambert, Nicolas S., 2019. "Proper scoring rules with general preferences: A dual characterization of optimal reports," Games and Economic Behavior, Elsevier, vol. 117(C), pages 322-341.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tsakas, Elias, 2020. "Robust scoring rules," Theoretical Economics, Econometric Society, vol. 15(3), July.
    2. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    3. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    4. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    5. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    6. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    7. Matějka, Filip & Mackowiak, Bartosz & Wiederholt, Mirko, 2018. "Survey: Rational Inattention, a Disciplined Behavioral Model," CEPR Discussion Papers 13243, C.E.P.R. Discussion Papers.
    8. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    9. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    10. Markus Eyting & Patrick Schmidt, 2019. "Belief Elicitation with Multiple Point Predictions," Working Papers 1818, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz, revised 16 Nov 2020.
    11. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    12. Arthur Carvalho & Stanko Dimitrov & Kate Larson, 2018. "On proper scoring rules and cumulative prospect theory," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 6(3), pages 343-376, November.
    13. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    14. Luciano Pomatto & Philipp Strack & Omer Tamuz, 2018. "The Cost of Information: The Case of Constant Marginal Costs," Papers 1812.04211, arXiv.org, revised Feb 2023.
    15. Li Hao & Daniel Houser, 2012. "Belief elicitation in the presence of naïve respondents: An experimental study," Journal of Risk and Uncertainty, Springer, vol. 44(2), pages 161-180, April.
    16. Burdea, Valeria & Woon, Jonathan, 2022. "Online belief elicitation methods," Journal of Economic Psychology, Elsevier, vol. 90(C).
    17. de Haan, Thomas, 2020. "Eliciting belief distributions using a random two-level partitioning of the state space," Working Papers in Economics 1/20, University of Bergen, Department of Economics.
    18. Kai Barron, 2021. "Belief updating: does the ‘good-news, bad-news’ asymmetry extend to purely financial domains?," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 31-58, March.
    19. Stefan T. Trautmann & Gijs Kuilen, 2015. "Belief Elicitation: A Horse Race among Truth Serums," Economic Journal, Royal Economic Society, vol. 125(589), pages 2116-2135, December.
    20. Guillaume Hollard & Sébastien Massoni & Jean-Christophe Vergnaud, 2016. "In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments," Theory and Decision, Springer, vol. 80(3), pages 363-387, March.

    More about this item

    JEL classification:

    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D87 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Neuroeconomics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:unm:umagsb:2018023. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Andrea Willems or Leonne Portz (email available below). General contact details of provider: https://edirc.repec.org/data/meteonl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.