IDEAS home Printed from https://ideas.repec.org/p/clt/sswopa/1271.html
   My bibliography  Save this paper

On eliciting beliefs in strategic games

Author

Listed:
  • Palfrey, Thomas R.
  • Wang, Stephanie W.

Abstract

Several recent studies in experimental economics have tried to measure beliefs of subjects engaged in strategic games with other subjects. Using data from one such study we conduct an experiment where our experienced subjects observe early rounds of strategy choices from that study and are given monetary incentives to report forecasts of choices in later rounds. We elicit beliefs using three different scoring rules: linear, logarithmic, and quadratic. We compare forecasts across the scoring rules and compare the forecasts of our trained observers to forecasts of the actual players in the original experiment. We find significant differences across scoring rules. The improper linear scoring rule produces forecasts closer to 0 and 1 than the proper rules, and these forecasts are poorly calibrated. The two proper scoring rules induce significantly different distributions of forecasts. We find that forecasts by observers under both proper scoring rules are significantly different from the forecasts of the actual players, in terms of accuracy, calibration, and the distribution of forecasts. We also find evidence for belief convergence among the observers.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Palfrey, Thomas R. & Wang, Stephanie W., "undated". "On eliciting beliefs in strategic games," Working Papers 1271, California Institute of Technology, Division of the Humanities and Social Sciences.
  • Handle: RePEc:clt:sswopa:1271
    as

    Download full text from publisher

    File URL: http://www.hss.caltech.edu/SSPapers/sswp1271R.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    2. McKelvey, Richard D. & Palfrey, Thomas R. & Weber, Roberto A., 2000. "The effects of payoff magnitude and heterogeneity on behavior in 2 x 2 games with unique mixed strategy equilibria," Journal of Economic Behavior & Organization, Elsevier, vol. 42(4), pages 523-548, August.
    3. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    4. McKelvey, Richard D & Page, Talbot, 1986. "Common Knowledge, Consensus, and Aggregate Information," Econometrica, Econometric Society, vol. 54(1), pages 109-127, January.
    5. Nielsen, Lars Tyge, et al, 1990. "Common Knowledge of an Aggregate of Expectations," Econometrica, Econometric Society, vol. 58(5), pages 1235-1239, September.
    6. Dominitz, Jeff & Hung, Angela A., 2009. "Empirical models of discrete choice and belief updating in observational learning experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 94-109, February.
    7. Dufwenberg, Martin & Gneezy, Uri, 2000. "Measuring Beliefs in an Experimental Lost Wallet Game," Games and Economic Behavior, Elsevier, vol. 30(2), pages 163-182, February.
    8. Huck, Steffen & Weizsacker, Georg, 2002. "Do players correctly estimate what others do? : Evidence of conservatism in beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 47(1), pages 71-85, January.
    9. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    10. Ochs Jack, 1995. "Games with Unique, Mixed Strategy Equilibria: An Experimental Study," Games and Economic Behavior, Elsevier, vol. 10(1), pages 202-217, July.
    11. Goeree, Jacob K. & Holt, Charles A. & Palfrey, Thomas R., 2003. "Risk averse behavior in generalized matching pennies games," Games and Economic Behavior, Elsevier, vol. 45(1), pages 97-113, October.
    12. Croson, Rachel T. A., 2000. "Thinking like a game theorist: factors affecting the frequency of equilibrium play," Journal of Economic Behavior & Organization, Elsevier, vol. 41(3), pages 299-314, March.
    13. Offerman, Theo & Sonnemans, Joep, 1998. "Learning by experience and learning by imitating successful others," Journal of Economic Behavior & Organization, Elsevier, vol. 34(4), pages 559-575, March.
    14. McKelvey, Richard D & Page, Talbot, 1990. "Public and Private Information: An Experimental Study of Information Pooling," Econometrica, Econometric Society, vol. 58(6), pages 1321-1339, November.
    15. Offerman, Theo & Sonnemans, Joep & Schram, Arthur, 1996. "Value Orientations, Expectations and Voluntary Contributions in Public Goods," Economic Journal, Royal Economic Society, vol. 106(437), pages 817-845, July.
    16. Ledyard, John & Hanson, Robin & Ishikida, Takashi, 2009. "An experimental test of combinatorial information markets," Journal of Economic Behavior & Organization, Elsevier, vol. 69(2), pages 182-189, February.
    17. Robert G. Nelson & David A. Bessler, 1989. "Subjective Probabilities and Scoring Rules: Experimental Evidence," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 71(2), pages 363-369.
    18. Robert L. Winkler, 1968. "The Consensus of Subjective Probability Distributions," Management Science, INFORMS, vol. 15(2), pages 61-75, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sandra Ludwig & Julia Nafziger, 2011. "Beliefs about overconfidence," Theory and Decision, Springer, vol. 70(4), pages 475-500, April.
    2. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    3. Costa-Gomes, Miguel A. & Huck, Steffen & Weizsäcker, Georg, 2014. "Beliefs and actions in the trust game: Creating instrumental variables to estimate the causal effect," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 88, pages 298-309.
    4. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    5. Miguel A. Costa-Gomes & Georg Weizsäcker, 2008. "Stated Beliefs and Play in Normal-Form Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 729-762.
    6. Lejarraga, Tomás & Lucena, Abel & Rubí-Barceló, Antoni, 2020. "Beliefs estimated from choices in Proposer-Responder Games," Journal of Economic Behavior & Organization, Elsevier, vol. 179(C), pages 442-459.
    7. Hoffmann, Timo, 2014. "The Effect of Belief Elicitation Game Play," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100483, Verein für Socialpolitik / German Economic Association.
    8. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    9. Frederic Koessler & Ch. Noussair & A. Ziegelmeyer, 2005. "Individual Behavior and Beliefs in Experimental Parimutuel Betting Markets," THEMA Working Papers 2005-08, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    10. Polonio, Luca & Coricelli, Giorgio, 2019. "Testing the level of consistency between choices and beliefs in games using eye-tracking," Games and Economic Behavior, Elsevier, vol. 113(C), pages 566-586.
    11. Frédéric Koessler & Charles Noussair & Anthony Ziegelmeyer, 2007. "Information Aggregation and Beliefs in Experimental Parimutuel Betting Markets," Papers on Strategic Interaction 2005-12, Max Planck Institute of Economics, Strategic Interaction Group.
    12. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    13. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    14. Rutstrom, E. Elizabet & Wilcox, Nathaniel, 2008. "Stated versus inferred beliefs: A methodological inquiry and experimental test," MPRA Paper 11852, University Library of Munich, Germany.
    15. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    16. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    17. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    18. Anna Conte & M. Levati, 2014. "Use of data on planned contributions and stated beliefs in the measurement of social preferences," Theory and Decision, Springer, vol. 76(2), pages 201-223, February.
    19. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    20. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:clt:sswopa:1271. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Victoria Mason (email available below). General contact details of provider: http://www.hss.caltech.edu/ss .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.