Advanced Search
MyIDEAS: Login to save this paper or follow this series

Opting-In: Participation Biases in the Lab

Contents:

Author Info

  • Slonim, Robert

    ()
    (University of Sydney)

  • Wang, Carmen

    ()
    (University of Sydney)

  • Garbarino, Ellen

    ()
    (University of Sydney)

  • Merrett, Danielle

    ()
    (University of Sydney)

Abstract

Assuming individuals rationally decide whether to participate or not to participate in lab experiments, we hypothesize several non-representative biases in the characteristics of lab participants. We test the hypotheses by first collecting survey and experimental data on a typical recruitment population and then inviting them to participate in a lab experiment. The results indicate that lab participants are not representative of the target population on almost all the hypothesized characteristics, including having lower income, working fewer hours, volunteering more often, and exhibiting behaviors correlated with interest in experiments and economics. We discuss the implications and various methods for addressing non-representative biases.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://ftp.iza.org/dp6865.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 6865.

as in new window
Length: 47 pages
Date of creation: Sep 2012
Date of revision:
Publication status: published in: Journal of Economic Behavior and Organization, 2013, 90, 43-70
Handle: RePEc:iza:izadps:dp6865

Contact details of provider:
Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information:
Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:

Related research

Keywords: laboratory experiments; participation bias;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Nagel, Rosemarie, 1995. "Unraveling in Guessing Games: An Experimental Study," American Economic Review, American Economic Association, American Economic Association, vol. 85(5), pages 1313-26, December.
  2. Burks, Stephen & Carpenter, Jeffrey & Goette, Lorenz, 2009. "Performance pay and worker cooperation: Evidence from an artefactual field experiment," Journal of Economic Behavior & Organization, Elsevier, Elsevier, vol. 70(3), pages 458-469, June.
  3. Matthias Benz & Stephan Meier, 2008. "Do people behave in experiments as in the field?—evidence from donations," Experimental Economics, Springer, Springer, vol. 11(3), pages 268-281, September.
  4. Hans Binswanger, 1981. "Attitudes toward risk: Theoretical implications of an experiment in rural india," Artefactual Field Experiments 00010, The Field Experiments Website.
  5. Werner Güth & Carsten Schmidt & Matthias Sutter, 2007. "Bargaining outside the lab - a newspaper experiment of a three-person ultimatum game," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 117(518), pages 449-469, 03.
  6. Samuel Bowles & Sandra Polania-Reyes, 2012. "Economic Incentives and Social Preferences: Substitutes or Complements?," Journal of Economic Literature, American Economic Association, vol. 50(2), pages 368-425, June.
  7. Anderson, Jon E. & Burks, Stephen V. & Carpenter, Jeffrey P. & Götte, Lorenz & Maurer, Karsten & Nosenzo, Daniele & Potter, Ruth & Rocha, Kim & Rustichini, Aldo, 2010. "Self Selection Does Not Increase Other-Regarding Preferences among Adult Laboratory Subjects, but Student Subjects May Be More Self-Regarding than Adults," IZA Discussion Papers 5389, Institute for the Study of Labor (IZA).
  8. Remoundou, Kyriaki & Drichoutis, Andreas & Koundouri, Phoebe, 2010. "Warm glow in charitable auctions: Are the WEIRDos driving the results?," MPRA Paper 25553, University Library of Munich, Germany.
  9. Bellemare, Charles & Kröger, Sabine, 2004. "On Representative Social Capital," IZA Discussion Papers 1145, Institute for the Study of Labor (IZA).
  10. Andersen, Steffen & Harrison, Glenn W. & Lau, Morten Igel & Rutström, E. Elisabet, 2010. "Preference heterogeneity in experiments: Comparing the field and laboratory," Journal of Economic Behavior & Organization, Elsevier, Elsevier, vol. 73(2), pages 209-224, February.
  11. Ben Greiner, 2004. "The Online Recruitment System ORSEE 2.0 - A Guide for the Organization of Experiments in Economics," Working Paper Series in Economics, University of Cologne, Department of Economics 10, University of Cologne, Department of Economics.
  12. Heckman, James J, 1979. "Sample Selection Bias as a Specification Error," Econometrica, Econometric Society, Econometric Society, vol. 47(1), pages 153-61, January.
  13. Bellemare, C. & Kroger, S., 2003. "On Representative Trust," Discussion Paper, Tilburg University, Center for Economic Research 2003-47, Tilburg University, Center for Economic Research.
  14. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
  15. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, American Economic Association, vol. 92(5), pages 1644-1655, December.
  16. John A. List, 2004. "Young, Selfish and Male: Field evidence of social preferences," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 114(492), pages 121-149, 01.
  17. Grether, David M & Plott, Charles R, 1979. "Economic Theory of Choice and the Preference Reversal Phenomenon," American Economic Review, American Economic Association, American Economic Association, vol. 69(4), pages 623-38, September.
  18. Hans Binswanger, 1980. "Attitudes toward risk: Experimental measurement in rural india," Artefactual Field Experiments 00009, The Field Experiments Website.
  19. Steven Levitt & John List, 2007. "Viewpoint: On the generalizability of lab behaviour to the field," Artefactual Field Experiments 00001, The Field Experiments Website.
  20. Machina, Mark J, 1987. "Choice under Uncertainty: Problems Solved and Unsolved," Journal of Economic Perspectives, American Economic Association, vol. 1(1), pages 121-54, Summer.
  21. Catherine Eckel & Philip Grossman, 2000. "Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments," Experimental Economics, Springer, Springer, vol. 3(2), pages 107-120, October.
  22. Ellen Garbarino & Robert Slonim & Justin Sydnor, 2011. "Digit ratios (2D:4D) as predictors of risky decision making for both sexes," Journal of Risk and Uncertainty, Springer, Springer, vol. 42(1), pages 1-26, February.
  23. Carpenter, Jeffrey P. & Burks, Stephen V. & Verhoogen, Eric, 2004. "Comparing Students to Workers: The Effects of Social Framing on Behavior in Distribution Games," IZA Discussion Papers 1341, Institute for the Study of Labor (IZA).
  24. John List, 2006. "The behavioralist meets the market: Measuring social preferences and reputation effects in actual transactions," Natural Field Experiments 00300, The Field Experiments Website.
  25. James Andreoni & Charles Sprenger, 2012. "Estimating Time Preferences from Convex Budgets," American Economic Review, American Economic Association, American Economic Association, vol. 102(7), pages 3333-56, December.
  26. Kahneman, Daniel & Tversky, Amos, 1979. "Prospect Theory: An Analysis of Decision under Risk," Econometrica, Econometric Society, Econometric Society, vol. 47(2), pages 263-91, March.
  27. Charles Bellemare & Sabine Kröger & Arthur van Soest, 2008. "Measuring Inequity Aversion in a Heterogeneous Population Using Experimental Decisions and Subjective Probabilities," Econometrica, Econometric Society, Econometric Society, vol. 76(4), pages 815-839, 07.
  28. John List & David Reiley, 2008. "Field experiments," Artefactual Field Experiments 00091, The Field Experiments Website.
  29. Garbarino, Ellen & Slonim, Robert, 2009. "The robustness of trust and reciprocity across a heterogeneous U.S. population," Journal of Economic Behavior & Organization, Elsevier, Elsevier, vol. 69(3), pages 226-240, March.
  30. Ernst Fehr & John A. List, 2004. "The Hidden Costs and Returns of Incentives-Trust and Trustworthiness Among CEOs," Journal of the European Economic Association, MIT Press, MIT Press, vol. 2(5), pages 743-771, 09.
  31. Falk, Armin & Meier, Stephan & Zehnder, Christian, 2011. "Did We Overestimate the Role of Social Preferences? The Case of Self-Selected Student Samples," IZA Discussion Papers 5475, Institute for the Study of Labor (IZA).
  32. Catherine C. Eckel & Philip J. Grossman, 2008. "Forecasting Risk Attitudes: An Experimental Study Using Actual and Forecast Gamble Choices," Development Research Unit Working Paper Series, Monash University, Department of Economics archive-01, Monash University, Department of Economics.
  33. Omar Al-Ubaydli & John A. List, 2012. "On the Generalizability of Experimental Results in Economics," NBER Working Papers 17957, National Bureau of Economic Research, Inc.
  34. Bellemare, Charles & Kröger, Sabine & van Soest, Arthur, 2011. "Preferences, intentions, and expectation violations: A large-scale experiment with a representative subject pool," Journal of Economic Behavior & Organization, Elsevier, Elsevier, vol. 78(3), pages 349-365, May.
  35. Juan Cardenas, 2011. "Social Norms and Behavior in the Local Commons as Seen Through the Lens of Field Experiments," Environmental & Resource Economics, European Association of Environmental and Resource Economists, European Association of Environmental and Resource Economists, vol. 48(3), pages 451-485, March.
  36. Eckel, Catherine C. & Grossman, Philip J., 1996. "Altruism in Anonymous Dictator Games," Games and Economic Behavior, Elsevier, Elsevier, vol. 16(2), pages 181-191, October.
  37. Blair L. Cleave & Nikos Nikiforakis & Robert Slonim, 2010. "Is There Selection Bias in Laboratory Experiments?," Department of Economics - Working Papers Series, The University of Melbourne 1106, The University of Melbourne.
  38. Axel Ockenfels & Gary E. Bolton, 2000. "ERC: A Theory of Equity, Reciprocity, and Competition," American Economic Review, American Economic Association, American Economic Association, vol. 90(1), pages 166-193, March.
  39. Jeffrey Carpenter & Cristina Connolly & Caitlin Myers, 2008. "Altruistic behavior in a representative dictator experiment," Experimental Economics, Springer, Springer, vol. 11(3), pages 282-298, September.
  40. Samuel Bowles & Sandra Polania-Reyes, 2011. "Economic incentives and social preferences: substitutes or complements?," Department of Economics University of Siena, Department of Economics, University of Siena 617, Department of Economics, University of Siena.
  41. repec:kap:expeco:v:11:y:2008:i:2:p:134-153 is not listed on IDEAS
  42. Tversky, Amos & Kahneman, Daniel, 1992. " Advances in Prospect Theory: Cumulative Representation of Uncertainty," Journal of Risk and Uncertainty, Springer, Springer, vol. 5(4), pages 297-323, October.
  43. Vivi Alatas & Lisa Cameron & Ananish Chaudhuri & Nisvan Erkal & Lata Gangadharan, 2006. "Subject Pool Effects in a Corruption Experiment: A Comparison of Indonesian Public Servants and Indonesian Students," Department of Economics - Working Papers Series, The University of Melbourne 975, The University of Melbourne.
  44. Stahl, Dale O., 1996. "Boundedly Rational Rule Learning in a Guessing Game," Games and Economic Behavior, Elsevier, Elsevier, vol. 16(2), pages 303-330, October.
  45. Michèle Belot & Raymond Duch & Luis Miller, 2010. "Who should be called to the lab? A comprehensive comparison of students and non-students in classic experimental games," Discussion Papers, University of Oxford, Nuffield College 2010001, University of Oxford, Nuffield College.
  46. R. Lynn Hannan & John H. Kagel & Donald V. Moser, 2002. "Partial Gift Exchange in an Experimental Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort Requests on Behavior," Journal of Labor Economics, University of Chicago Press, University of Chicago Press, vol. 20(4), pages 923-951, October.
  47. Cameron, Lisa & Chaudhuri, Ananish & Erkal, Nisvan & Gangadharan, Lata, 2009. "Propensities to engage in and punish corrupt behavior: Experimental evidence from Australia, India, Indonesia and Singapore," Journal of Public Economics, Elsevier, Elsevier, vol. 93(7-8), pages 843-851, August.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Cleave, Blair L. & Nikiforakis, Nikos & Slonim, Robert, 2011. "Is There Selection Bias in Laboratory Experiments? The Case of Social and Risk Preferences," IZA Discussion Papers 5488, Institute for the Study of Labor (IZA).
  2. Omar Al-Ubaydli & John A. List, 2013. "On the Generalizability of Experimental Results in Economics: With A Response To Camerer," NBER Working Papers 19666, National Bureau of Economic Research, Inc.
  3. Mirco Tonin & Michael Vlassopoulos, 2014. "Corporate Philanthropy and Productivity: Evidence from an Online Real Effort Experiment," CESifo Working Paper Series 4778, CESifo Group Munich.
  4. Abeler, Johannes & Nosenzo, Daniele, 2013. "Self-Selection into Economics Experiments Is Driven by Monetary Rewards," IZA Discussion Papers 7374, Institute for the Study of Labor (IZA).
  5. Judd Kessler, 2013. "When will there be Gift Exchange? Addressing the Lab-Field Debate with Laboratory Gift Exchange Experiments," CESifo Working Paper Series 4161, CESifo Group Munich.
  6. Omar Al-Ubaydli & John A. List, 2013. "On the Generalizability of Experimental Results in Economics: With a Response to Commentors," CESifo Working Paper Series 4543, CESifo Group Munich.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp6865. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.