IDEAS home Printed from https://ideas.repec.org/p/not/notcdx/2013-03.html
   My bibliography  Save this paper

Self-selection into Economics Experiments is Driven by Monetary Rewards

Author

Listed:
  • Johannes Abeler

    (School of Economics, University of Oxford)

  • Daniele Nosenzo

    (School of Economics, University of Nottingham)

Abstract

Laboratory experiments have become a wide-spread tool in economic research. Yet, there is still doubt about how well the results from lab experiments generalize to other settings. In this paper, we investigate the self-selection process of potential subjects into the subject pool. We alter the recruitment email sent to first-year students, either mentioning the monetary reward associated with participation in experiments; or appealing to the importance of helping research; or both. We find that the sign-up rate drops by two-thirds if we do not mention monetary rewards. Appealing to subjects’ willingness to help research has no effect on signup. We then invite the so-recruited subjects to the laboratory to measure a range of preferences in incentivized experiments. We do not find any differences between the three groups. Our results show that student subjects participate in experiments foremost to earn money, and that it is therefore unlikely that this selection leads to an over-estimation of social preferences in the student population.

Suggested Citation

  • Johannes Abeler & Daniele Nosenzo, 2013. "Self-selection into Economics Experiments is Driven by Monetary Rewards," Discussion Papers 2013-03, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
  • Handle: RePEc:not:notcdx:2013-03
    as

    Download full text from publisher

    File URL: https://www.nottingham.ac.uk/cedex/documents/papers/cedex-discussion-paper-2013-03.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Burks, Stephen & Carpenter, Jeffrey & Goette, Lorenz, 2009. "Performance pay and worker cooperation: Evidence from an artefactual field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 70(3), pages 458-469, June.
    2. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    3. Sonnemans, Joep & Schram, Arthur & Offerman, Theo, 1998. "Public good provision and public bad prevention: The effect of framing," Journal of Economic Behavior & Organization, Elsevier, vol. 34(1), pages 143-161, January.
    4. Catherine C. Eckel & Philip J. Grossman, 2000. "Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments," Experimental Economics, Springer;Economic Science Association, vol. 3(2), pages 107-120, October.
    5. Blair Cleave & Nikos Nikiforakis & Robert Slonim, 2013. "Is there selection bias in laboratory experiments? The case of social and risk preferences," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 372-382, September.
    6. Hoffman, Elizabeth & McCabe, Kevin & Smith, Vernon L, 1996. "Social Distance and Other-Regarding Behavior in Dictator Games," American Economic Review, American Economic Association, vol. 86(3), pages 653-660, June.
    7. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    8. Cubitt, Robin P. & Drouvelis, Michalis & Gächter, Simon & Kabalin, Ruslan, 2011. "Moral judgments in social dilemmas: How bad is free riding?," Journal of Public Economics, Elsevier, vol. 95(3), pages 253-264.
    9. Kei Tsutsui & Daniel Zizzo, 2014. "Group status, minorities and trust," Experimental Economics, Springer;Economic Science Association, vol. 17(2), pages 215-244, June.
    10. Thomas Dohmen & Armin Falk, 2010. "You Get What You Pay For: Incentives and Selection in the Education System," Economic Journal, Royal Economic Society, vol. 120(546), pages 256-271, August.
    11. Armin Falk & James J. Heckman, 2009. "Lab Experiments are a Major Source of Knowledge in the Social Sciences," CESifo Working Paper Series 2894, CESifo.
    12. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    13. Hans-Martin Gaudecker & Arthur Soest & Erik Wengström, 2012. "Experts in experiments," Journal of Risk and Uncertainty, Springer, vol. 45(2), pages 159-190, October.
    14. Thomas Dohmen & Armin Falk, 2011. "Performance Pay and Multidimensional Sorting: Productivity, Preferences, and Gender," American Economic Review, American Economic Association, vol. 101(2), pages 556-590, April.
    15. James Banks & Zoe Oldfield, 2007. "Understanding Pensions: Cognitive Function, Numerical Ability and Retirement Saving," Fiscal Studies, Institute for Fiscal Studies, vol. 28(2), pages 143-170, June.
    16. Urs Fischbacher & Simon Gachter, 2010. "Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Goods Experiments," American Economic Review, American Economic Association, vol. 100(1), pages 541-556, March.
    17. Park, Eun-Soo, 2000. "Warm-glow versus cold-prickle: a further experimental study of framing effects on free-riding," Journal of Economic Behavior & Organization, Elsevier, vol. 43(4), pages 405-421, December.
    18. Muller, Laurent & Sefton, Martin & Steinberg, Richard & Vesterlund, Lise, 2008. "Strategic behavior and learning in repeated voluntary contribution experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 67(3-4), pages 782-793, September.
    19. Benedikt Herrmann & Christian Thöni, 2009. "Measuring conditional cooperation: a replication study in Russia," Experimental Economics, Springer;Economic Science Association, vol. 12(1), pages 87-92, March.
    20. Bruno S. Frey & Iris Bohnet, 1999. "Social Distance and Other-Regarding Behavior in Dictator Games: Comment," American Economic Review, American Economic Association, vol. 89(1), pages 335-339, March.
    21. Kocher, Martin G. & Cherry, Todd & Kroll, Stephan & Netzer, Robert J. & Sutter, Matthias, 2008. "Conditional cooperation on three continents," Economics Letters, Elsevier, vol. 101(3), pages 175-178, December.
    22. Roberto Burlando & Francesco Guala, 2005. "Heterogeneous Agents in Public Goods Experiments," Experimental Economics, Springer;Economic Science Association, vol. 8(1), pages 35-54, April.
    23. Neil Buckley & Kenneth Chan & James Chowhan & Stuart Mestelman & Mohamed Shehata, 2001. "Value Orientations, Income and Displacement Effects, and Voluntary Contributions," Experimental Economics, Springer;Economic Science Association, vol. 4(2), pages 183-195, October.
    24. Jon Anderson & Stephen Burks & Jeffrey Carpenter & Lorenz Götte & Karsten Maurer & Daniele Nosenzo & Ruth Potter & Kim Rocha & Aldo Rustichini, 2013. "Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: evidence from one college student and two adult samples," Experimental Economics, Springer;Economic Science Association, vol. 16(2), pages 170-189, June.
    25. Fischbacher, Urs & Gachter, Simon & Fehr, Ernst, 2001. "Are people conditionally cooperative? Evidence from a public goods experiment," Economics Letters, Elsevier, vol. 71(3), pages 397-404, June.
    26. repec:feb:artefa:00073 is not listed on IDEAS
    27. List John A., 2007. "Field Experiments: A Bridge between Lab and Naturally Occurring Data," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(2), pages 1-47, April.
    28. Edward L. Glaeser & David I. Laibson & José A. Scheinkman & Christine L. Soutter, 2000. "Measuring Trust," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(3), pages 811-846.
      • Glaeser, Edward Ludwig & Laibson, David I. & Scheinkman, Jose A. & Soutter, Christine L., 2000. "Measuring Trust," Scholarly Articles 4481497, Harvard University Department of Economics.
    29. Slonim, Robert & Wang, Carmen & Garbarino, Ellen & Merrett, Danielle, 2012. "Opting-In: Participation Biases in the Lab," IZA Discussion Papers 6865, Institute of Labor Economics (IZA).
    30. Jenni, Karen E & Loewenstein, George, 1997. "Explaining the "Identifiable Victim Effect."," Journal of Risk and Uncertainty, Springer, vol. 14(3), pages 235-257, May-June.
    31. Jan Stoop & Charles N. Noussair & Daan van Soest, 2012. "From the Lab to the Field: Cooperation among Fishermen," Journal of Political Economy, University of Chicago Press, vol. 120(6), pages 1027-1056.
    32. Offerman, Theo & Sonnemans, Joep & Schram, Arthur, 1996. "Value Orientations, Expectations and Voluntary Contributions in Public Goods," Economic Journal, Royal Economic Society, vol. 106(437), pages 817-845, July.
    33. Charness, Gary & Gneezy, Uri, 2008. "What's in a name? Anonymity and social distance in dictator and ultimatum games," Journal of Economic Behavior & Organization, Elsevier, vol. 68(1), pages 29-35, October.
    34. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    35. Michal Krawczyk, 2011. "What brings your subjects to the lab? A field experiment," Natural Field Experiments 00694, The Field Experiments Website.
    36. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    37. Michèle Belot & Raymond Duch & Luis Miller, 2010. "Who should be called to the lab? A comprehensive comparison of students and non-students in classic experimental games," Discussion Papers 2010001, University of Oxford, Nuffield College.
    38. Greiner, Ben, 2004. "An Online Recruitment System for Economic Experiments," MPRA Paper 13513, University Library of Munich, Germany.
    39. Michal Krawczyk, 2011. "What brings your subjects to the lab? A field experiment," Experimental Economics, Springer;Economic Science Association, vol. 14(4), pages 482-489, November.
    40. M. Vittoria Levati & Matteo Ploner & Stefan Traub, 2011. "Are conditional cooperators willing to forgo efficiency gains? Evidence from a public goods experiment," New Zealand Economic Papers, Taylor & Francis Journals, vol. 45(1-2), pages 47-57.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Are there biases from monetary rewards in experimental economics?
      by Economic Logician in Economic Logic on 2013-08-28 19:58:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kölle, Felix & Gächter, Simon & Quercia, Simone, 2014. "The ABC of Cooperation in Voluntary Contribution and Common Pool Extraction Games," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100417, Verein für Socialpolitik / German Economic Association.
    2. Guillen, Pablo & Hing, Alexander, 2014. "Lying through their teeth: Third party advice and truth telling in a strategy proof mechanism," European Economic Review, Elsevier, vol. 70(C), pages 178-185.
    3. Gogi, Anastasia & Tako, Antuela A. & Robinson, Stewart, 2016. "An experimental investigation into the role of simulation models in generating insights," European Journal of Operational Research, Elsevier, vol. 249(3), pages 931-944.
    4. John, Katrin & Thomsen, Stephan L., 2015. "School-track environment or endowment: What determines different other-regarding behavior across peer groups?," Games and Economic Behavior, Elsevier, vol. 94(C), pages 122-141.
    5. Pelligra, Vittorio & Stanca, Luca, 2013. "To give or not to give? Equity, efficiency and altruistic behavior in an artefactual field experiment," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 46(C), pages 1-9.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johannes Abeler & Daniele Nosenzo, 2015. "Self-selection into laboratory experiments: pro-social motives versus monetary incentives," Experimental Economics, Springer;Economic Science Association, vol. 18(2), pages 195-214, June.
    2. Jon Anderson & Stephen Burks & Jeffrey Carpenter & Lorenz Götte & Karsten Maurer & Daniele Nosenzo & Ruth Potter & Kim Rocha & Aldo Rustichini, 2013. "Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: evidence from one college student and two adult samples," Experimental Economics, Springer;Economic Science Association, vol. 16(2), pages 170-189, June.
    3. Fischbacher, Urs & Gächter, Simon & Quercia, Simone, 2012. "The behavioral validity of the strategy method in public good experiments," Journal of Economic Psychology, Elsevier, vol. 33(4), pages 897-913.
    4. Matteo M. Galizzi & Daniel Navarro-Martinez, 2019. "On the External Validity of Social Preference Games: A Systematic Lab-Field Study," Management Science, INFORMS, vol. 65(3), pages 976-1002, March.
    5. Schmidt, Robert & Schwieren, Christiane & Sproten, Alec N., 2020. "Norms in the lab: Inexperienced versus experienced participants," Journal of Economic Behavior & Organization, Elsevier, vol. 173(C), pages 239-255.
    6. Fischbacher, Urs & Gächter, Simon & Quercia, Simone, 2012. "The behavioral validity of the strategy method in public good experiments," Journal of Economic Psychology, Elsevier, vol. 33(4), pages 897-913.
    7. Gächter, Simon & Thöni, Christian, 2010. "Social comparison and performance: Experimental evidence on the fair wage-effort hypothesis," Journal of Economic Behavior & Organization, Elsevier, vol. 76(3), pages 531-543, December.
    8. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    9. Kölle, Felix & Gächter, Simon & Quercia, Simone, 2014. "The ABC of Cooperation in Voluntary Contribution and Common Pool Extraction Games," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100417, Verein für Socialpolitik / German Economic Association.
    10. Frijters, Paul & Kong, Tao Sherry & Liu, Elaine M., 2015. "Who is coming to the artefactual field experiment? Participation bias among Chinese rural migrants," Journal of Economic Behavior & Organization, Elsevier, vol. 114(C), pages 62-74.
    11. Slonim, Robert & Wang, Carmen & Garbarino, Ellen & Merrett, Danielle, 2013. "Opting-in: Participation bias in economic experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 43-70.
    12. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    13. Weimann, Joachim & Brosig-Koch, Jeannette & Heinrich, Timo & Hennig-Schmidt, Heike & Keser, Claudia, 2022. "CO2 Emission reduction – Real public good provision by large groups in the laboratory," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 1076-1089.
    14. Boosey, Luke & Mark Isaac, R. & Norton, Douglas & Stinn, Joseph, 2020. "Cooperation, contributor types, and control questions," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    15. Schmidt, Robert J. & Schwieren, Christiane & Sproten, Alec N., 2018. "Social Norm Perception in Economic Laboratory Experiments: Inexperienced versus Experienced Participants," Working Papers 0656, University of Heidelberg, Department of Economics.
    16. Urs Fischbacher & Simon Gaechter, 2008. "Heterogeneous Social Preferences And The Dynamics Of Free Riding In Public Good Experiments," Discussion Papers 2008-07, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    17. Stephen Cheung, 2014. "New insights into conditional cooperation and punishment from a strategy method experiment," Experimental Economics, Springer;Economic Science Association, vol. 17(1), pages 129-153, March.
    18. Barmettler, Franziska & Fehr, Ernst & Zehnder, Christian, 2012. "Big experimenter is watching you! Anonymity and prosocial behavior in the laboratory," Games and Economic Behavior, Elsevier, vol. 75(1), pages 17-34.
    19. Volk, Stefan & Thöni, Christian & Ruigrok, Winfried, 2012. "Temporal stability and psychological foundations of cooperation preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 81(2), pages 664-676.
    20. Grandjean, Gilles & Lefebvre, Mathieu & Mantovani, Marco, 2022. "Preferences and strategic behavior in public goods games," Journal of Economic Behavior & Organization, Elsevier, vol. 195(C), pages 171-185.

    More about this item

    Keywords

    Methodology; Selection bias; Laboratory experiment; Field experiment; Otherregarding behavior; Social preferences; Social Approval; Experimenter Demand.;
    All these keywords.

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • D03 - Microeconomics - - General - - - Behavioral Microeconomics: Underlying Principles

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:not:notcdx:2013-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jose V Guinot Saporta (email available below). General contact details of provider: https://edirc.repec.org/data/cdnotuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.