IDEAS home Printed from https://ideas.repec.org/p/feb/artefa/00397.html
   My bibliography  Save this paper

Control in Experiments: A Simple Model

Author

Listed:
  • Omar Al-Ubaydli
  • John List

Abstract

A commonly held view is that laboratory experiments provide researchers with more "control" than natural field experiments, and that this advantage is to be balanced against the disadvantage that laboratory experiments are less generalizable. This paper presents a simple model that explores circumstances under which natural field experiments provide researchers with more control than laboratory experiments afford. This stems from the covertness of natural field experiments: laboratory experiments provide researchers with a high degree of control in the environment which participants agree to be experimental subjects. When participants systematically opt out of laboratory experiments, the researcher's ability to manipulate certain variables is limited. In contrast, natural field experiments bypass the participation decision altogether and allow for a potentially more diverse participant pool within the market of interest. We show one particular case where such selection is invaluable: when treatment effects interact with participant characteristics.

Suggested Citation

  • Omar Al-Ubaydli & John List, 2015. "Control in Experiments: A Simple Model," Artefactual Field Experiments 00397, The Field Experiments Website.
  • Handle: RePEc:feb:artefa:00397
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00397.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. List John A. & Millimet Daniel L, 2008. "The Market: Catalyst for Rationality and Filter of Irrationality," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(1), pages 1-55, November.
    2. Colin F. Camerer, 1998. "Can Asset Markets Be Manipulated? A Field Experiment with Racetrack Betting," Journal of Political Economy, University of Chicago Press, vol. 106(3), pages 457-482, June.
    3. John A. List, 2006. "Using Hicksian Surplus Measures to Examine Consistency of Individual Preferences: Evidence from a Field Experiment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 108(1), pages 115-134, March.
    4. Smith, Vernon L & Walker, James M, 1993. "Monetary Rewards and Decision Cost in Experimental Economics," Economic Inquiry, Western Economic Association International, vol. 31(2), pages 245-261, April.
    5. John A. List, 2007. "On the Interpretation of Giving in Dictator Games," Journal of Political Economy, University of Chicago Press, vol. 115, pages 482-493.
    6. List, John A. & Shogren, Jason F., 1998. "Calibration of the difference between actual and hypothetical valuations in a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 37(2), pages 193-205, October.
    7. repec:feb:framed:0081 is not listed on IDEAS
    8. John A. List & Daniel L. Millimet & Per G. Fredriksson & W. Warren McHone, 2003. "Effects of Environmental Regulations on Manufacturing Plant Births: Evidence from a Propensity Score Matching Estimator," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 944-952, November.
    9. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    10. Castillo, Marco & Petrie, Ragan & Torero, Maximo & Vesterlund, Lise, 2013. "Gender differences in bargaining outcomes: A field experiment on discrimination," Journal of Public Economics, Elsevier, vol. 99(C), pages 35-48.
    11. John A. List, 2004. "Testing Neoclassical Competitive Theory in Multilateral Decentralized Markets," Journal of Political Economy, University of Chicago Press, vol. 112(5), pages 1131-1156, October.
    12. Glenn W. Harrison & Morten I. Lau & E. Elisabet Rutström, 2007. "Estimating Risk Attitudes in Denmark: A Field Experiment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 109(2), pages 341-368, June.
    13. List, John A. & Cherry, Todd L., 2008. "Examining the role of fairness in high stakes allocation decisions," Journal of Economic Behavior & Organization, Elsevier, vol. 65(1), pages 1-8, January.
    14. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    15. John A. List, 2001. "Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards," American Economic Review, American Economic Association, vol. 91(5), pages 1498-1507, December.
    16. John List & Jason Shogren, 1998. "The Deadweight Loss from Christmas: Comment," Artefactual Field Experiments 00531, The Field Experiments Website.
    17. repec:feb:framed:0070 is not listed on IDEAS
    18. John List, 2005. "Scientific Numerology, Preference Anomalies, and Environmental Policymaking," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 32(1), pages 35-53, September.
    19. John A. List, 2009. "Social Preferences: Some Thoughts from the Field," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 563-583, May.
    20. John A. List, 2004. "Young, Selfish and Male: Field evidence of social preferences," Economic Journal, Royal Economic Society, vol. 114(492), pages 121-149, January.
    21. List, John A & Shogren, Jason F, 1998. "The Deadweight Loss of Christmas: Comment," American Economic Review, American Economic Association, vol. 88(5), pages 1350-1355, December.
    22. repec:feb:artefa:0091 is not listed on IDEAS
    23. John List, 2008. "Homo experimentalis evolves," Artefactual Field Experiments 00084, The Field Experiments Website.
    24. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    25. Colin Camerer, 1998. "Can asset markets be manipulated? A field experiment with racetrack betting," Natural Field Experiments 00222, The Field Experiments Website.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Haghani, Milad & Sarvi, Majid, 2019. "Laboratory experimentation and simulation of discrete direction choices: Investigating hypothetical bias, decision-rule effect and external validity based on aggregate prediction measures," Transportation Research Part A: Policy and Practice, Elsevier, vol. 130(C), pages 134-157.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    2. John A. List, 2014. "Using Field Experiments to Change the Template of How We Teach Economics," The Journal of Economic Education, Taylor & Francis Journals, vol. 45(2), pages 81-89, June.
    3. Jeffrey Flory & Uri Gneezy & Kenneth Leonard & John List, 2012. "Sex, competitiveness, and investment in offspring: On the origin of preferences," Artefactual Field Experiments 00072, The Field Experiments Website.
    4. Omar Al-Ubaydli & John A. List, 2015. "Do Natural Field Experiments Afford Researchers More or Less Control than Laboratory Experiments? A Simple Model," NBER Working Papers 20877, National Bureau of Economic Research, Inc.
    5. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    6. Omar Al-Ubaydli & John List, 2016. "Field Experiments in Markets," Artefactual Field Experiments j0002, The Field Experiments Website.
    7. Matteo M. Galizzi & Daniel Navarro-Martinez, 2019. "On the External Validity of Social Preference Games: A Systematic Lab-Field Study," Management Science, INFORMS, vol. 65(3), pages 976-1002, March.
    8. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    9. List John A., 2007. "Field Experiments: A Bridge between Lab and Naturally Occurring Data," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(2), pages 1-47, April.
    10. John List, 2021. "2021 Summary Data of Artefactual Field Experiments Published on Fieldexperiments.com," Artefactual Field Experiments 00749, The Field Experiments Website.
    11. Mohammed H. Alemu & Søren B. Olsen, 2017. "Can a Repeated Opt-Out Reminder remove hypothetical bias in discrete choice experiments? An application to consumer valuation of novel food products," IFRO Working Paper 2017/05, University of Copenhagen, Department of Food and Resource Economics.
    12. John List, 2022. "2021 Summary Data of Natural Field Experiments Published on Fieldexperiments.com," Natural Field Experiments 00747, The Field Experiments Website.
    13. John List, 2022. "Framed Field Experiments: 2021 Summary on Fieldexperiments.com," Framed Field Experiments 00752, The Field Experiments Website.
    14. Omar Al-Ubaydli & John List, 2013. "On the Generalizability of Experimental Results in Economics: With A Response To Camerer," Artefactual Field Experiments j0001, The Field Experiments Website.
    15. Brit Grosskopf & Graeme Pearce, 2020. "Do You Mind Me Paying Less? Measuring Other-Regarding Preferences in the Market for Taxis," Management Science, INFORMS, vol. 66(11), pages 5059-5074, November.
    16. Francisco B. Galarza, 2017. "Trust and Trustworthiness in College: An Experimental Analysis," Working Papers 17-03, Centro de Investigación, Universidad del Pacífico.
    17. Stephen V. Burks & Daniele Nosenzo & Jon Anderson & Matthew Bombyk & Derek Ganzhorn & Lorenz Goette & Aldo Rustichini, 2015. "Lab Measures of Other-Regarding Preferences Can Predict Some Related on-the-Job Behavior: Evidence from a Large Scale Field Experiment," Discussion Papers 2015-21, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    18. Pelligra, Vittorio & Stanca, Luca, 2013. "To give or not to give? Equity, efficiency and altruistic behavior in an artefactual field experiment," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 46(C), pages 1-9.
    19. Jan Stoop, 2014. "From the lab to the field: envelopes, dictators and manners," Experimental Economics, Springer;Economic Science Association, vol. 17(2), pages 304-313, June.
    20. John A. List & Ragan Petrie & Anya Samek, 2023. "How Experiments with Children Inform Economics," Journal of Economic Literature, American Economic Association, vol. 61(2), pages 504-564, June.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:artefa:00397. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: David Franks (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.