IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article or follow this journal

Opting-in: Participation bias in economic experiments

Author Info

  • Slonim, Robert
  • Wang, Carmen
  • Garbarino, Ellen
  • Merrett, Danielle

Abstract

Assuming individuals rationally decide whether to participate or not to participate in lab experiments, we hypothesize several non-representative biases in the characteristics of lab participants. We test the hypotheses by first collecting survey and experimental data from a typical recruitment population and then inviting them to participate in a lab experiment. The results indicate that lab participants are not representative of the target population on almost all the hypothesized characteristics, including having lower income, working fewer hours, volunteering more often, and exhibiting behaviors correlated with interest in experiments and economics. The results reinforce the commonly understood limits of laboratory research to make quantitative inferences. We also discuss several methods for addressing non-representative biases to advance laboratory methods for improving quantitative inferences and consequently increasing confidence in qualitative conclusions.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/pii/S0167268113000589
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Journal of Economic Behavior & Organization.

Volume (Year): 90 (2013)
Issue (Month): C ()
Pages: 43-70

as
in new window

Handle: RePEc:eee:jeborg:v:90:y:2013:i:c:p:43-70
Contact details of provider: Web page: http://www.elsevier.com/locate/jebo

Keywords: Participation bias; Laboratory experiments;

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Hans Binswanger, 1981. "Attitudes toward risk: Theoretical implications of an experiment in rural india," Artefactual Field Experiments 00010, The Field Experiments Website.
  2. Catherine C. Eckel & Philip J. Grossman, 2008. "Forecasting Risk Attitudes: An Experimental Study Using Actual and Forecast Gamble Choices," Monash Economics Working Papers archive-01, Monash University, Department of Economics.
  3. Roth, Alvin E. & Vesna Prasnikar & Masahiro Okuno-Fujiwara & Shmuel Zamir, 1991. "Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study," American Economic Review, American Economic Association, vol. 81(5), pages 1068-95, December.
  4. John List & Steven Levitt, 2007. "Viewpoint: On the generalizability of lab behaviour to the field," Artefactual Field Experiments 00001, The Field Experiments Website.
  5. Harrison, Glenn W. & Lau, Morten I. & Elisabet Rutström, E., 2009. "Risk attitudes, randomization to treatment, and self-selection into experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 70(3), pages 498-507, June.
  6. Smith, Vernon L, 1976. "Experimental Economics: Induced Value Theory," American Economic Review, American Economic Association, vol. 66(2), pages 274-79, May.
  7. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
  8. Carsten Schmidt & Matthias Sutter & Werner Guth, 2002. "Bargaining outside the lab - a newspaper experiment of a three person-ultimatum game," Artefactual Field Experiments 00050, The Field Experiments Website.
  9. Eckel, Catherine C. & Grossman, Philip J., 1996. "Altruism in Anonymous Dictator Games," Games and Economic Behavior, Elsevier, vol. 16(2), pages 181-191, October.
  10. James Andreoni & Charles Sprenger, 2012. "Estimating Time Preferences from Convex Budgets," American Economic Review, American Economic Association, vol. 102(7), pages 3333-56, December.
  11. Falk, Armin & Heckman, James J., 2009. "Lab Experiments Are a Major Source of Knowledge in the Social Sciences," IZA Discussion Papers 4540, Institute for the Study of Labor (IZA).
  12. Bellemare, Charles & Kröger, Sabine, 2004. "On Representative Social Capital," IZA Discussion Papers 1145, Institute for the Study of Labor (IZA).
  13. Bellemare, Charles & Kröger, Sabine, 2003. "On Representative Trust," SFB 373 Discussion Papers 2003,24, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
  14. Andreas Drichoutis & Kyriaki Remoundou & Phoebe Koundouri, . "Warm glow in charitable auctions: Are the WEIRDos driving the results?," DEOS Working Papers 1025, Athens University of Economics and Business.
  15. Matthias Benz & Stephan Meier, . "Do People Behave in Experiments as in the Field? – Evidence from Donations," IEW - Working Papers 248, Institute for Empirical Research in Economics - University of Zurich.
  16. Hans Binswanger, 1980. "Attitudes toward risk: Experimental measurement in rural india," Artefactual Field Experiments 00009, The Field Experiments Website.
  17. Samuel Bowles & Sandra Polania-Reyes, 2012. "Economic Incentives and Social Preferences: Substitutes or Complements?," Journal of Economic Literature, American Economic Association, vol. 50(2), pages 368-425, June.
  18. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer, vol. 14(3), pages 399-425, September.
  19. repec:feb:artefa:00073 is not listed on IDEAS
  20. Ernst Fehr & Joseph Henrich & Robert Boyd, 2003. "In Search of Homo Economicus: Behavioral Experiments in 15 Small- Scale Societies," Microeconomics 0305009, EconWPA.
  21. Charles Bellemare & Sabine Kröger & Arthur van Soest, 2008. "Measuring Inequity Aversion in a Heterogeneous Population Using Experimental Decisions and Subjective Probabilities," Econometrica, Econometric Society, vol. 76(4), pages 815-839, 07.
  22. David Reiley & John List, 2008. "Field experiments," Artefactual Field Experiments 00091, The Field Experiments Website.
  23. Edward L. Glaeser & David I. Laibson & José A. Scheinkman & Christine L. Soutter, 2000. "Measuring Trust," The Quarterly Journal of Economics, MIT Press, vol. 115(3), pages 811-846, August.
    • Glaeser, Edward Ludwig & Laibson, David I. & Scheinkman, Jose A. & Soutter, Christine L., 2000. "Measuring Trust," Scholarly Articles 4481497, Harvard University Department of Economics.
  24. Vivi Alatas & Lisa Cameron & Ananish Chaudhuri & Nisvan Erkal & Lata Gangadharan, 2009. "Subject pool effects in a corruption experiment: A comparison of Indonesian public servants and Indonesian students," Experimental Economics, Springer, vol. 12(1), pages 113-132, March.
  25. Baran, Nicole M. & Sapienza, Paola & Zingales, Luigi, 2010. "Can we infer social preferences from the lab? Evidence from the trust game," CEPR Discussion Papers 7634, C.E.P.R. Discussion Papers.
  26. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
  27. John List, 2004. "Young, selfish, and male: Field evidence of social preferences," Natural Field Experiments 00298, The Field Experiments Website.
  28. Michal Krawczyk, 2011. "What brings your subjects to the lab? A field experiment," Experimental Economics, Springer, vol. 14(4), pages 482-489, November.
  29. R. Lynn Hannan & John H. Kagel & Donald V. Moser, 2002. "Partial Gift Exchange in an Experimental Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort Requests on Behavior," Journal of Labor Economics, University of Chicago Press, vol. 20(4), pages 923-951, October.
  30. Marco Casari & John C. Ham & John H. Kagel, 2005. "Selection bias, demographic effects, and ability effects in common value auction experiments," Staff Reports 213, Federal Reserve Bank of New York.
  31. John A. List, 2005. "The Behavioralist Meets the Market: Measuring Social Preferences and Reputation Effects in Actual Transactions," NBER Working Papers 11616, National Bureau of Economic Research, Inc.
  32. Ellen Garbarino & Robert Slonim & Justin Sydnor, 2011. "Digit ratios (2D:4D) as predictors of risky decision making for both sexes," Journal of Risk and Uncertainty, Springer, vol. 42(1), pages 1-26, February.
  33. Michèle Belot & Raymond Duch & Luis Miller, 2010. "Who should be called to the lab? A comprehensive comparison of students and non-students in classic experimental games," Discussion Papers 2010001, University of Oxford, Nuffield College.
  34. Burks, Stephen & Carpenter, Jeffrey & Goette, Lorenz, 2009. "Performance pay and worker cooperation: Evidence from an artefactual field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 70(3), pages 458-469, June.
  35. Garbarino, Ellen & Slonim, Robert, 2009. "The robustness of trust and reciprocity across a heterogeneous U.S. population," Journal of Economic Behavior & Organization, Elsevier, vol. 69(3), pages 226-240, March.
  36. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
  37. Catherine Eckel & Philip Grossman, 2000. "Volunteers and Pseudo-Volunteers: The Effect of Recruitment Method in Dictator Experiments," Experimental Economics, Springer, vol. 3(2), pages 107-120, October.
  38. Roe, Brian E. & Haab, Timothy C. & Beversdorf, David Q. & Gu, Howard H. & Tilley, Michael R., 2009. "Risk-attitude selection bias in subject pools for experiments involving neuroimaging and blood samples," Journal of Economic Psychology, Elsevier, vol. 30(2), pages 181-189, April.
  39. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," NBER Working Papers 6699, National Bureau of Economic Research, Inc.
  40. David J. Cooper, 1999. "Gaming against Managers in Incentive Systems: Experimental Results with Chinese Students and Chinese Managers," American Economic Review, American Economic Association, vol. 89(4), pages 781-804, September.
  41. Cameron, Lisa & Chaudhuri, Ananish & Erkal, Nisvan & Gangadharan, Lata, 2009. "Propensities to engage in and punish corrupt behavior: Experimental evidence from Australia, India, Indonesia and Singapore," Journal of Public Economics, Elsevier, vol. 93(7-8), pages 843-851, August.
  42. Jeffrey Carpenter & Cristina Connolly & Caitlin Myers, 2008. "Altruistic behavior in a representative dictator experiment," Experimental Economics, Springer, vol. 11(3), pages 282-298, September.
  43. Juan Cardenas, 2011. "Social Norms and Behavior in the Local Commons as Seen Through the Lens of Field Experiments," Environmental & Resource Economics, European Association of Environmental and Resource Economists, vol. 48(3), pages 451-485, March.
  44. Carpenter, Jeffrey P. & Burks, Stephen V. & Verhoogen, Eric, 2004. "Comparing Students to Workers: The Effects of Social Framing on Behavior in Distribution Games," IZA Discussion Papers 1341, Institute for the Study of Labor (IZA).
  45. Bellemare, C. & Kroger, S. & van Soest, A.H.O., 2008. "Measuring inequity aversion in a heterogeneous population using experimental decisions and subjective probabilities," Other publications TiSEM f17bda32-98f9-4580-ab3f-6, School of Economics and Management.
  46. Omar Al-Ubaydli & John A. List, 2012. "On the Generalizability of Experimental Results in Economics," NBER Working Papers 17957, National Bureau of Economic Research, Inc.
Full references (including those not matched with items on IDEAS)

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

as in new window


Cited by:

  1. Atalay, Kadir & Bakhtiar, Fayzan & Cheung, Stephen L. & Slonim, Robert, 2012. "Savings and Prize-Linked Savings Accounts," IZA Discussion Papers 6927, Institute for the Study of Labor (IZA).
  2. Matteo M. Galizzi & Daniel Navarro-Martínez, 2015. "On the External Validity of Social- Preference Games: A Systematic Lab-Field Study," Working Papers 802, Barcelona Graduate School of Economics.
  3. Mirco Tonin & Michael Vlassopoulos, 2014. "Corporate Philanthropy and Productivity: Evidence from an Online Real Effort Experiment," CESifo Working Paper Series 4778, CESifo Group Munich.

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Access and download statistics

When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:90:y:2013:i:c:p:43-70. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.