IDEAS home Printed from https://ideas.repec.org/p/jrp/jrpwrp/2010-021.html
   My bibliography  Save this paper

Applying Quadratic Scoring Rule transparently in multiple choice settings: A note

Author

Listed:
  • Florian Artinger

    (Max Planck Institute for Human Development and Technical University Berlin)

  • Filippos Exadaktylos

    (University of Granada)

  • Hannes Koppel

    (Max Planck Institute of Economics, Jena)

  • Lauri Sääksvuori

    (Max Planck Institute of Economics, Jena)

Abstract

The quadratic scoring rule (QSR) is often used to guarantee an incentive compatible elicitation of subjective probabilities over events. Experimentalists have regularly not been able to ensure that subjects fully comprehend the consequences of their actions on payoffs given the rules of the games. In this note, we present a procedure that allows the transparent use of the QSR even in multiple-choice scenarios. For that purpose, two methodological means are applied: an alternative representation of the score and a short learning period to familiarize subjects with the payoff mechanism. The results suggest that both means were necessary and successful in facilitating subjects' understanding of the rule.

Suggested Citation

  • Florian Artinger & Filippos Exadaktylos & Hannes Koppel & Lauri Sääksvuori, 2010. "Applying Quadratic Scoring Rule transparently in multiple choice settings: A note," Jena Economics Research Papers 2010-021, Friedrich-Schiller-University Jena.
    • Handle: RePEc:jrp:jrpwrp:2010-021
    as

    Download full text from publisher

    File URL: https://oweb.b67.uni-jena.de/Papers/jerp2010/wp_2010_021.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    2. Steffen Andersen & John Fountain & Glenn Harrison & E. Rutström, 2014. "Estimating subjective probabilities," Journal of Risk and Uncertainty, Springer, vol. 48(3), pages 207-229, June.
    3. Smith, Vernon L & Walker, James M, 1993. "Monetary Rewards and Decision Cost in Experimental Economics," Economic Inquiry, Western Economic Association International, vol. 31(2), pages 245-261, April.
    4. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    5. Paul J. Ferraro & Ronald G. Cummings, 2007. "Cultural Diversity, Discrimination, And Economic Outcomes: An Experimental Analysis," Economic Inquiry, Western Economic Association International, vol. 45(2), pages 217-232, April.
    6. Neugebauer, Tibor & Perote, Javier & Schmidt, Ulrich & Loos, Malte, 2009. "Selfish-biased conditional cooperation: On the decline of contributions in repeated public goods experiments," Journal of Economic Psychology, Elsevier, vol. 30(1), pages 52-60, February.
    7. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    8. David Danz & Dietmar Fehr & Dorothea Kübler, 2012. "Information and beliefs in a repeated normal-form game," Experimental Economics, Springer;Economic Science Association, vol. 15(4), pages 622-640, December.
    9. Guarino, Antonio & Huck, Steffen & Jeitschko, Thomas D., 2006. "Averting economic collapse and the solipsism bias," Games and Economic Behavior, Elsevier, vol. 57(2), pages 264-285, November.
    10. Ana Leon-Mejia & Luis M. Miller, 2007. "The Devil is in the Details - Sex Differences in Simple Bargaining Games," Jena Economics Research Papers 2007-069, Friedrich-Schiller-University Jena.
    11. Croson, Rachel T. A., 1999. "The Disjunction Effect and Reason-Based Choice in Games, , , , , , , , , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 80(2), pages 118-133, November.
    12. Palfrey, Thomas R. & Wang, Stephanie W., 2009. "On eliciting beliefs in strategic games," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 98-109, August.
    13. Daniel Read, 2005. "Monetary incentives, what are they good for?," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 265-276.
    14. Smith, Vernon L, 1976. "Experimental Economics: Induced Value Theory," American Economic Review, American Economic Association, vol. 66(2), pages 274-279, May.
    15. Erev, Ido & Bornstein, Gary & Wallsten, Thomas S., 1993. "The Negative Effect of Probability Assessments on Decision Quality," Organizational Behavior and Human Decision Processes, Elsevier, vol. 55(1), pages 78-94, June.
    16. Dufwenberg, Martin & Gneezy, Uri, 2000. "Measuring Beliefs in an Experimental Lost Wallet Game," Games and Economic Behavior, Elsevier, vol. 30(2), pages 163-182, February.
    17. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    18. Wolf Ze'ev Ehrblatt & Kyle Hyndman & Erkut Y. ÄOzbay & Andrew Schotter, 2006. "Convergence: An Experimental Study," Levine's Working Paper Archive 122247000000001148, David K. Levine.
    19. Croson, Rachel T. A., 2000. "Thinking like a game theorist: factors affecting the frequency of equilibrium play," Journal of Economic Behavior & Organization, Elsevier, vol. 41(3), pages 299-314, March.
    20. Andrew Schotter & Barry Sopher, 2006. "Trust and trustworthiness in games: An experimental study of intergenerational advice," Experimental Economics, Springer;Economic Science Association, vol. 9(2), pages 123-145, June.
    21. Rey-Biel, Pedro, 2009. "Equilibrium play and best response to (stated) beliefs in normal form games," Games and Economic Behavior, Elsevier, vol. 65(2), pages 572-585, March.
    22. Irwin, Julie R, et al, 1998. "Payoff Dominance vs. Cognitive Transparency in Decision Making," Economic Inquiry, Western Economic Association International, vol. 36(2), pages 272-285, April.
    23. Offerman, Theo & Sonnemans, Joep & Schram, Arthur, 1996. "Value Orientations, Expectations and Voluntary Contributions in Public Goods," Economic Journal, Royal Economic Society, vol. 106(437), pages 817-845, July.
    24. Andreas Ortmann & John Fitzgerald & Carl Boeing, 2000. "Trust, Reciprocity, and Social History: A Re-examination," Experimental Economics, Springer;Economic Science Association, vol. 3(1), pages 81-100, June.
    25. Eileen Chou & Margaret McConnell & Rosemarie Nagel & Charles Plott, 2009. "The control of game form recognition in experiments: understanding dominant strategy failures in a simple two person “guessing” game," Experimental Economics, Springer;Economic Science Association, vol. 12(2), pages 159-179, June.
    26. Olivier Armantier & Nicolas Treich, 2009. "Subjective Probabilities In Games: An Application To The Overbidding Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1079-1102, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vera Popva, 2010. "What renders financial advisors less treacherous? - On commissions and reciprocity -," Jena Economics Research Papers 2010-036, Friedrich-Schiller-University Jena.
    2. Dimant, Eugen, 2023. "Beyond average: A method for measuring the tightness, looseness, and polarization of social norms," Economics Letters, Elsevier, vol. 233(C).
    3. Bracht, Jürgen & Regner, Tobias, 2013. "Moral emotions and partnership," Journal of Economic Psychology, Elsevier, vol. 39(C), pages 313-326.
    4. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    5. Luciano Andreozzi & Matteo Ploner & Ali Seyhun Saral, 2019. "The Stability of Conditional Cooperation: Egoism Trumps Reciprocity in Social Dilemmas," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2019_12, Max Planck Institute for Research on Collective Goods.
    6. Fairley, Kim & Parelman, Jacob M. & Jones, Matt & Carter, R. McKell, 2019. "Risky health choices and the Balloon Economic Risk Protocol," Journal of Economic Psychology, Elsevier, vol. 73(C), pages 15-33.
    7. Astrid Matthey & Tobias Regner, 2011. "More than outcomes: A cognitive dissonance-based explanation of other-regarding behavior," Jena Economics Research Papers 2011-024, Friedrich-Schiller-University Jena.
    8. Eugen Dimant & Michele Gelfand & Anna Hochleitner & Silvia Sonderegger, 2022. "Strategic Behavior with Tight, Loose and Polarized Norms," ECONtribute Discussion Papers Series 198, University of Bonn and University of Cologne, Germany.
    9. Adriana Breaban & Charles N. Noussair & Andreea Victoria Popescu, 2018. "Your money or your time? Experimental evidence on overbidding in all-pay auctions," Working Papers 18-20, Chapman University, Economic Science Institute.
    10. Angelova, Vera & Regner, Tobias, 2013. "Do voluntary payments to advisors improve the quality of financial advice? An experimental deception game," Journal of Economic Behavior & Organization, Elsevier, vol. 93(C), pages 205-218.
    11. Popescu, Andreea Victoria, 2020. "Essays in asset pricing and auctions," Other publications TiSEM 879f7643-7123-4bc8-a5e7-6, Tilburg University, School of Economics and Management.
    12. Eugen Dimant & Michele Gelfand & Anna Hochleitner & Silvia Sonderegger, 2023. "Strategic Behavior with Tight, Loose and Polarized Norms," CESifo Working Paper Series 10233, CESifo.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    2. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    3. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    4. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    5. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    6. Armantier, Olivier & Treich, Nicolas, 2013. "Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging," European Economic Review, Elsevier, vol. 62(C), pages 17-40.
    7. Polonio, Luca & Coricelli, Giorgio, 2019. "Testing the level of consistency between choices and beliefs in games using eye-tracking," Games and Economic Behavior, Elsevier, vol. 113(C), pages 566-586.
    8. Claudia Neri, 2015. "Eliciting beliefs in continuous-choice games: a double auction experiment," Experimental Economics, Springer;Economic Science Association, vol. 18(4), pages 569-608, December.
    9. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    10. Charness, Gary & Gneezy, Uri & Halladay, Brianna, 2016. "Experimental methods: Pay one or pay all," Journal of Economic Behavior & Organization, Elsevier, vol. 131(PA), pages 141-150.
    11. Popescu, Andreea Victoria, 2020. "Essays in asset pricing and auctions," Other publications TiSEM 879f7643-7123-4bc8-a5e7-6, Tilburg University, School of Economics and Management.
    12. Adriana Breaban & Charles N. Noussair & Andreea Victoria Popescu, 2018. "Your money or your time? Experimental evidence on overbidding in all-pay auctions," Working Papers 18-20, Chapman University, Economic Science Institute.
    13. Hoffmann, Timo, 2014. "The Effect of Belief Elicitation Game Play," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100483, Verein für Socialpolitik / German Economic Association.
    14. Eyting, Markus & Schmidt, Patrick, 2021. "Belief elicitation with multiple point predictions," European Economic Review, Elsevier, vol. 135(C).
    15. Sandra Ludwig & Julia Nafziger, 2011. "Beliefs about overconfidence," Theory and Decision, Springer, vol. 70(4), pages 475-500, April.
    16. Stefan T. Trautmann & Gijs Kuilen, 2015. "Belief Elicitation: A Horse Race among Truth Serums," Economic Journal, Royal Economic Society, vol. 125(589), pages 2116-2135, December.
    17. Folli, Dominik & Wolff, Irenaeus, 2022. "Biases in belief reports," Journal of Economic Psychology, Elsevier, vol. 88(C).
    18. Manski, Charles F. & Neri, Claudia, 2013. "First- and second-order subjective expectations in strategic decision-making: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 81(C), pages 232-254.
    19. Costa-Gomes, Miguel A. & Huck, Steffen & Weizsäcker, Georg, 2014. "Beliefs and actions in the trust game: Creating instrumental variables to estimate the causal effect," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 88, pages 298-309.
    20. David Danz & Dietmar Fehr & Dorothea Kübler, 2012. "Information and beliefs in a repeated normal-form game," Experimental Economics, Springer;Economic Science Association, vol. 15(4), pages 622-640, December.

    More about this item

    Keywords

    Quadratic scoring rule; experimental methodology; experimental design;
    All these keywords.

    JEL classification:

    • D84 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Expectations; Speculations
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jrp:jrpwrp:2010-021. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Markus Pasche (email available below). General contact details of provider: http://www.jenecon.de .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.