IDEAS home Printed from https://ideas.repec.org/p/exc/wpaper/2025-03.html

Salient Rewards, Payoff Protocols and Biased Data

Author

Listed:
  • James C. Cox
  • Vjollca Sadiraj

Abstract

Paying salient rewards to subjects is expensive. Is it worth it? We describe previous data from otherwise identical treatments with hypothetical or salient payoffs that indicate opposite conclusions about the central research question in a paper. If we are going to pay salient rewards, how should we? The answer depends on the theoretical model underlying hypotheses being tested with the data, hence the choice of payoff protocol is an essential feature of experimental design. In recent years there has been renewed interest in theoretical and experimental research on properties of payoff protocols. We present an extension and critical discussion of the literature intended to promote uptake.

Suggested Citation

  • James C. Cox & Vjollca Sadiraj, 2025. "Salient Rewards, Payoff Protocols and Biased Data," Experimental Economics Center Working Paper Series 2025-03, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    • Handle: RePEc:exc:wpaper:2025-03
    as

    Download full text from publisher

    File URL: http://excen.gsu.edu/workingpapers/GSU_EXCEN_WP_2025-03.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yi Li, 2021. "The ABC mechanism: an incentive compatible payoff mechanism for elicitation of outcome and probability transformations," Experimental Economics, Springer;Economic Science Association, vol. 24(3), pages 1019-1046, September.
    2. Marina Agranov & Pietro Ortoleva, 2022. "Revealed Preferences for Randomization: An Overview," AEA Papers and Proceedings, American Economic Association, vol. 112, pages 426-430, May.
    3. Smith, Vernon L, 1982. "Microeconomic Systems as an Experimental Science," American Economic Review, American Economic Association, vol. 72(5), pages 923-955, December.
    4. Robin Cubitt & Chris Starmer & Robert Sugden, 1998. "On the Validity of the Random Lottery Incentive System," Experimental Economics, Springer;Economic Science Association, vol. 1(2), pages 115-131, September.
    5. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    6. David J. Freeman & Yoram Halevy & Terri Kneeland, 2019. "Eliciting risk preferences using choice lists," Quantitative Economics, Econometric Society, vol. 10(1), pages 217-237, January.
    7. James C. Cox & Vjollca Sadiraj & Ulrich Schmidt, 2012. "Asymmetrically Dominated Choice Problems and Random Incentive Mechanisms," Experimental Economics Center Working Paper Series 2012-10, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University, revised Mar 2014.
    8. James Cox & Vjollca Sadiraj & Ulrich Schmidt, 2015. "Paradoxes and mechanisms for choice under risk," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 215-250, June.
    9. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    10. Segal, Uzi, 1988. "Does the Preference Reversal Phenomenon Necessarily Contradict the Independence Axiom?," American Economic Review, American Economic Association, vol. 78(1), pages 233-236, March.
    11. Marina Agranov & Pietro Ortoleva, 2017. "Stochastic Choice and Preferences for Randomization," Journal of Political Economy, University of Chicago Press, vol. 125(1), pages 40-68.
    12. Richard H. Thaler & Eric J. Johnson, 1990. "Gambling with the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice," Management Science, INFORMS, vol. 36(6), pages 643-660, June.
    13. Quiggin, John, 1982. "A theory of anticipated utility," Journal of Economic Behavior & Organization, Elsevier, vol. 3(4), pages 323-343, December.
    14. Cox, James C & Epstein, Seth, 1989. "Preference Reversals without the Independence Axiom," American Economic Review, American Economic Association, vol. 79(3), pages 408-426, June.
    15. John Hey & Jinkwon Lee, 2005. "Do subjects remember the past?," Applied Economics, Taylor & Francis Journals, vol. 37(1), pages 9-18.
    16. Machina, Mark J, 1985. "Stochastic Choice Functions Generated from Deterministic Preferences over Lotteries," Economic Journal, Royal Economic Society, vol. 95(379), pages 575-594, September.
    17. John Hey & Jinkwon Lee, 2005. "Do Subjects Separate (or Are They Sophisticated)?," Experimental Economics, Springer;Economic Science Association, vol. 8(3), pages 233-265, September.
    18. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    19. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    20. David M. Grether & James C. Cox, 1996. "The preference reversal phenomenon: Response mode, markets and incentives (*)," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 7(3), pages 381-405.
    21. Holt, Charles A, 1986. "Preference Reversals and the Independence Axiom," American Economic Review, American Economic Association, vol. 76(3), pages 508-515, June.
    22. Segal, Uzi, 1990. "Two-Stage Lotteries without the Reduction Axiom," Econometrica, Econometric Society, vol. 58(2), pages 349-377, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2020. "Incentives in experiments with objective lotteries," Experimental Economics, Springer;Economic Science Association, vol. 23(1), pages 1-29, March.
    2. James Cox & Vjollca Sadiraj & Ulrich Schmidt, 2015. "Paradoxes and mechanisms for choice under risk," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 215-250, June.
    3. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    4. Drichoutis, Andreas C. & Palma, Marco & Feldman, Paul, 2024. "Incentives and Payment Mechanisms in Preference Elicitation," MPRA Paper 120898, University Library of Munich, Germany.
    5. Nathaniel T. Wilcox, 2024. "Conditional independence in a binary choice experiment," Journal of Risk and Uncertainty, Springer, vol. 68(3), pages 205-225, June.
    6. Guido Baltussen & G. Post & Martijn Assem & Peter Wakker, 2012. "Random incentive systems in a dynamic choice experiment," Experimental Economics, Springer;Economic Science Association, vol. 15(3), pages 418-443, September.
    7. Patrick DeJarnette & David Dillenberger & Daniel Gottlieb & Pietro Ortoleva, 2020. "Time Lotteries and Stochastic Impatience," Econometrica, Econometric Society, vol. 88(2), pages 619-656, March.
    8. Yi Li, 2021. "The ABC mechanism: an incentive compatible payoff mechanism for elicitation of outcome and probability transformations," Experimental Economics, Springer;Economic Science Association, vol. 24(3), pages 1019-1046, September.
    9. Patrick DeJarnette & David Dillenberger & Daniel Gottlieb & Pietro Ortoleva, 2014. "Time Lotteries, Second Version," PIER Working Paper Archive 15-026v2, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 12 Jan 2018.
    10. Cheung, Paul H.Y. & Ellis, Keaton, 2025. "Non-isolation, reversals, and social preference," Games and Economic Behavior, Elsevier, vol. 154(C), pages 159-174.
    11. James C. Cox & Vjollca Sadiraj, 2018. "Incentives," Experimental Economics Center Working Paper Series 2018-01, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    12. Uri Gneezy & Yoram Halevy & Brian Hall & Theo Offerman & Jeroen van de Ven, 2024. "How Real is Hypothetical? A High-Stakes Test of the Allais Paradox," Working Papers tecipa-783, University of Toronto, Department of Economics.
    13. Jinkwon Lee, 2008. "The effect of the background risk in a simple chance improving decision model," Journal of Risk and Uncertainty, Springer, vol. 36(1), pages 19-41, February.
    14. Ulrich Schmidt & Christian Seidl, 2014. "Reconsidering the common ratio effect: the roles of compound independence, reduction, and coalescing," Theory and Decision, Springer, vol. 77(3), pages 323-339, October.
    15. Glenn Harrison & J. Swarthout, 2014. "Experimental payment protocols and the Bipolar Behaviorist," Theory and Decision, Springer, vol. 77(3), pages 423-438, October.
    16. Ilke Aydogan & Loïc Berger & Valentina Bosetti & Ning Liu, 2023. "Three Layers of Uncertainty," Journal of the European Economic Association, European Economic Association, vol. 21(5), pages 2209-2236.
    17. Victor H. Gonzalez-Jimenez, 2019. "Contracting Probability Distortions," Vienna Economics Papers vie1901, University of Vienna, Department of Economics.
    18. Aurélien Baillon & Yoram Halevy & Chen Li, 2022. "Experimental elicitation of ambiguity attitude using the random incentive system," Experimental Economics, Springer;Economic Science Association, vol. 25(3), pages 1002-1023, June.
    19. Brown, Alexander L. & Healy, Paul J., 2018. "Separated decisions," European Economic Review, Elsevier, vol. 101(C), pages 20-34.
    20. David J. Freeman & Guy Mayraz, 2019. "Why choice lists increase risk taking," Experimental Economics, Springer;Economic Science Association, vol. 22(1), pages 131-154, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C7 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • D9 - Microeconomics - - Micro-Based Behavioral Economics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:exc:wpaper:2025-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: J. Todd Swarthout (email available below). General contact details of provider: https://edirc.repec.org/data/exgsuus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.