IDEAS home Printed from https://ideas.repec.org/p/exc/wpaper/2025-03.html

Salient Rewards, Payoff Protocols and Biased Data

Author

Listed:
  • James C. Cox
  • Vjollca Sadiraj

Abstract

Paying salient rewards to subjects is expensive. Is it worth it? We describe previous data from otherwise identical treatments with hypothetical or salient payoffs that indicate opposite conclusions about the central research question in a paper. If we are going to pay salient rewards, how should we? The answer depends on the theoretical model underlying hypotheses being tested with the data, hence the choice of payoff protocol is an essential feature of experimental design. In recent years there has been renewed interest in theoretical and experimental research on properties of payoff protocols. We present an extension and critical discussion of the literature intended to promote uptake.

Suggested Citation

  • James C. Cox & Vjollca Sadiraj, 2025. "Salient Rewards, Payoff Protocols and Biased Data," Experimental Economics Center Working Paper Series 2025-03, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University, revised Jul 2026.
    • Handle: RePEc:exc:wpaper:2025-03
    as

    Download full text from publisher

    File URL: http://excen.gsu.edu/workingpapers/GSU_EXCEN_WP_2025-03.pdf
    File Function: First version, 2025
    Download Restriction: no
    File URL: http://excen.gsu.edu/workingpapers/GSU_EXCEN_WP_2026-03.pdf
    File Function: Revised version, 2026
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yi Li, 2021. "The ABC mechanism: an incentive compatible payoff mechanism for elicitation of outcome and probability transformations," Experimental Economics, Springer;Economic Science Association, vol. 24(3), pages 1019-1046, September.
    2. Marina Agranov & Pietro Ortoleva, 2022. "Revealed Preferences for Randomization: An Overview," AEA Papers and Proceedings, American Economic Association, vol. 112, pages 426-430, May.
    3. Smith, Vernon L, 1982. "Microeconomic Systems as an Experimental Science," American Economic Review, American Economic Association, vol. 72(5), pages 923-955, December.
    4. Robin Cubitt & Chris Starmer & Robert Sugden, 1998. "On the Validity of the Random Lottery Incentive System," Experimental Economics, Springer;Economic Science Association, vol. 1(2), pages 115-131, September.
    5. Tversky, Amos & Kahneman, Daniel, 1992. "Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, vol. 5(4), pages 297-323, October.
    6. David J. Freeman & Yoram Halevy & Terri Kneeland, 2019. "Eliciting risk preferences using choice lists," Quantitative Economics, Econometric Society, vol. 10(1), pages 217-237, January.
    7. James C. Cox & Vjollca Sadiraj & Ulrich Schmidt, 2012. "Asymmetrically Dominated Choice Problems and Random Incentive Mechanisms," Experimental Economics Center Working Paper Series 2012-10, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University, revised Mar 2014.
    8. James Cox & Vjollca Sadiraj & Ulrich Schmidt, 2015. "Paradoxes and mechanisms for choice under risk," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 215-250, June.
    9. Starmer, Chris & Sugden, Robert, 1991. "Does the Random-Lottery Incentive System Elicit True Preferences? An Experimental Investigation," American Economic Review, American Economic Association, vol. 81(4), pages 971-978, September.
    10. Segal, Uzi, 1988. "Does the Preference Reversal Phenomenon Necessarily Contradict the Independence Axiom?," American Economic Review, American Economic Association, vol. 78(1), pages 233-236, March.
    11. Marina Agranov & Pietro Ortoleva, 2017. "Stochastic Choice and Preferences for Randomization," Journal of Political Economy, University of Chicago Press, vol. 125(1), pages 40-68.
    12. Richard H. Thaler & Eric J. Johnson, 1990. "Gambling with the House Money and Trying to Break Even: The Effects of Prior Outcomes on Risky Choice," Management Science, INFORMS, vol. 36(6), pages 643-660, June.
    13. Quiggin, John, 1982. "A theory of anticipated utility," Journal of Economic Behavior & Organization, Elsevier, vol. 3(4), pages 323-343, December.
    14. Cox, James C & Epstein, Seth, 1989. "Preference Reversals without the Independence Axiom," American Economic Review, American Economic Association, vol. 79(3), pages 408-426, June.
    15. John Hey & Jinkwon Lee, 2005. "Do subjects remember the past?," Applied Economics, Taylor & Francis Journals, vol. 37(1), pages 9-18.
    16. Machina, Mark J, 1985. "Stochastic Choice Functions Generated from Deterministic Preferences over Lotteries," Economic Journal, Royal Economic Society, vol. 95(379), pages 575-594, September.
    17. John Hey & Jinkwon Lee, 2005. "Do Subjects Separate (or Are They Sophisticated)?," Experimental Economics, Springer;Economic Science Association, vol. 8(3), pages 233-265, September.
    18. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    19. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    20. David M. Grether & James C. Cox, 1996. "The preference reversal phenomenon: Response mode, markets and incentives (*)," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 7(3), pages 381-405.
    21. Holt, Charles A, 1986. "Preference Reversals and the Independence Axiom," American Economic Review, American Economic Association, vol. 76(3), pages 508-515, June.
    22. Segal, Uzi, 1990. "Two-Stage Lotteries without the Reduction Axiom," Econometrica, Econometric Society, vol. 58(2), pages 349-377, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2020. "Incentives in experiments with objective lotteries," Experimental Economics, Springer;Economic Science Association, vol. 23(1), pages 1-29, March.
    2. James Cox & Vjollca Sadiraj & Ulrich Schmidt, 2015. "Paradoxes and mechanisms for choice under risk," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 215-250, June.
    3. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    4. Drichoutis, Andreas C. & Palma, Marco & Feldman, Paul, 2024. "Incentives and Payment Mechanisms in Preference Elicitation," MPRA Paper 120898, University Library of Munich, Germany.
    5. Nathaniel T. Wilcox, 2024. "Conditional independence in a binary choice experiment," Journal of Risk and Uncertainty, Springer, vol. 68(3), pages 205-225, June.
    6. Guido Baltussen & G. Post & Martijn Assem & Peter Wakker, 2012. "Random incentive systems in a dynamic choice experiment," Experimental Economics, Springer;Economic Science Association, vol. 15(3), pages 418-443, September.
    7. Patrick DeJarnette & David Dillenberger & Daniel Gottlieb & Pietro Ortoleva, 2020. "Time Lotteries and Stochastic Impatience," Econometrica, Econometric Society, vol. 88(2), pages 619-656, March.
    8. Yi Li, 2021. "The ABC mechanism: an incentive compatible payoff mechanism for elicitation of outcome and probability transformations," Experimental Economics, Springer;Economic Science Association, vol. 24(3), pages 1019-1046, September.
    9. Patrick DeJarnette & David Dillenberger & Daniel Gottlieb & Pietro Ortoleva, 2014. "Time Lotteries, Second Version," PIER Working Paper Archive 15-026v2, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 12 Jan 2018.
    10. Cheung, Paul H.Y. & Ellis, Keaton, 2025. "Non-isolation, reversals, and social preference," Games and Economic Behavior, Elsevier, vol. 154(C), pages 159-174.
    11. James C. Cox & Vjollca Sadiraj, 2018. "Incentives," Experimental Economics Center Working Paper Series 2018-01, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    12. Ulrich Schmidt & Christian Seidl, 2014. "Reconsidering the common ratio effect: the roles of compound independence, reduction, and coalescing," Theory and Decision, Springer, vol. 77(3), pages 323-339, October.
    13. Glenn Harrison & J. Swarthout, 2014. "Experimental payment protocols and the Bipolar Behaviorist," Theory and Decision, Springer, vol. 77(3), pages 423-438, October.
    14. Ilke Aydogan & Loïc Berger & Valentina Bosetti & Ning Liu, 2023. "Three Layers of Uncertainty," Journal of the European Economic Association, European Economic Association, vol. 21(5), pages 2209-2236.
    15. Victor H. Gonzalez-Jimenez, 2019. "Contracting Probability Distortions," Vienna Economics Papers vie1901, University of Vienna, Department of Economics.
    16. Brown, Alexander L. & Healy, Paul J., 2018. "Separated decisions," European Economic Review, Elsevier, vol. 101(C), pages 20-34.
    17. Patrick DeJarnette & David Dillenberger & Daniel Gottlieb & Pietro Ortoleva, 2015. "Time Lotteries," PIER Working Paper Archive 15-026, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 31 Jul 2015.
    18. Victor H. Gonzalez-Jimenez, 2019. "Contracting Probability Distortions," Vienna Economics Papers 1901, University of Vienna, Department of Economics.
    19. Hajimoladarvish, Narges, 2018. "How do people reduce compound lotteries?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 75(C), pages 126-133.
    20. Jinkwon Lee, 2007. "Repetition And Financial Incentives In Economics Experiments," Journal of Economic Surveys, Wiley Blackwell, vol. 21(3), pages 628-681, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C7 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • D9 - Microeconomics - - Micro-Based Behavioral Economics

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:exc:wpaper:2025-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: J. Todd Swarthout (email available below). General contact details of provider: https://edirc.repec.org/data/exgsuus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.