IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00804.html
   My bibliography  Save this paper

The Experimentalist Looks Within: Toward an Understanding of Within-Subject Experimental Designs

Author

Listed:
  • John List

Abstract

The traditional approach in experimental economics is to use a between-subject design: the analyst places each unit in treatment or control simultaneously and recovers outcome differences via differencing conditional expectations. Within-subject designs represent a significant departure from this method, as the same unit is observed in both treatment and control conditions sequentially. While some might consider the design choice straightforward (always opt for a between-subject design), I contend that researchers should meticulously weigh the advantages and disadvantages of each design. In doing so, I propose a categorization for within-subject designs based on the plausibility of recovering an internally valid estimate. In one instance, which I denote as stealth designs, the analyst should unequivocally choose a within-subject design rather than a between-subject design.

Suggested Citation

  • John List, 2025. "The Experimentalist Looks Within: Toward an Understanding of Within-Subject Experimental Designs," Natural Field Experiments 00804, The Field Experiments Website.
  • Handle: RePEc:feb:natura:00804
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00804.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rodemeier, Matthias, 2023. "Willingness to Pay for Carbon Mitigation: Field Evidence from the Market for Carbon Offsets," IZA Discussion Papers 15939, Institute of Labor Economics (IZA).
    2. Charles Bellemare & Luc Bissonnette & Sabine Kröger, 2014. "Statistical Power of Within and Between-Subjects Designs in Economic Experiments," CESifo Working Paper Series 5055, CESifo.
    3. Peter Hull, 2018. "Estimating Treatment Effects in Mover Designs," Papers 1804.06721, arXiv.org.
    4. John A. List, 2004. "The Nature and Extent of Discrimination in the Marketplace: Evidence from the Field," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 49-89.
    5. Uri Gneezy, 2005. "Deception: The Role of Consequences," American Economic Review, American Economic Association, vol. 95(1), pages 384-394, March.
    6. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jordan, Diana & Trexler, Andrew & Ollerenshaw, Trent, 2025. "New Evidence and Design Considerations for Repeated Measure Experiments in Survey Research," OSF Preprints q6czp_v1, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    2. David Masclet & David L. Dickinson, 2025. "Incorporating conditional morality into economic decisions," Theory and Decision, Springer, vol. 98(1), pages 95-152, February.
    3. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    4. Marie Claire Villeval, 2019. "Comportements (non) éthiques et stratégies morales," Revue économique, Presses de Sciences-Po, vol. 70(6), pages 1021-1046.
    5. Omar Al-Ubaydli & John A. List, 2019. "How natural field experiments have enhanced our understanding of unemployment," Nature Human Behaviour, Nature, vol. 3(1), pages 33-39, January.
    6. Ayelet Gneezy & Alex Imas & Amber Brown & Leif D. Nelson & Michael I. Norton, 2012. "Paying to Be Nice: Consistency and Costly Prosocial Behavior," Management Science, INFORMS, vol. 58(1), pages 179-187, January.
    7. Uyanga Turmunkh & Martijn J. van den Assem & Dennie van Dolder, 2019. "Malleable Lies: Communication and Cooperation in a High Stakes TV Game Show," Management Science, INFORMS, vol. 65(10), pages 4795-4812, October.
    8. John List, 2022. "2021 Summary Data of Natural Field Experiments Published on Fieldexperiments.com," Natural Field Experiments 00747, The Field Experiments Website.
    9. Steffen Andersen & Seda Ertac & Uri Gneezy & Moshe Hoffman & John A. List, 2011. "Stakes Matter in Ultimatum Games," American Economic Review, American Economic Association, vol. 101(7), pages 3427-3439, December.
    10. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    11. Hernández-Pizarro, Helena M. & Nicodemo, Catia & Casasnovas, Guillem López, 2020. "Discontinuous system of allowances: The response of prosocial health-care professionals," Journal of Public Economics, Elsevier, vol. 190(C).
    12. Dugar, Subhasish & Bhattacharya, Haimanti, 2017. "Fishy behavior: A field experiment on (dis)honesty in the marketplace," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 67(C), pages 41-55.
    13. Omar Al-Ubaydli & John List, 2016. "Field Experiments in Markets," Artefactual Field Experiments j0002, The Field Experiments Website.
    14. Rode, Julian, 2010. "Truth and trust in communication: Experiments on the effect of a competitive context," Games and Economic Behavior, Elsevier, vol. 68(1), pages 325-338, January.
    15. Zhixin Dai & Fabio Galeotti & Marie Claire Villeval, 2018. "Cheating in the Lab Predicts Fraud in the Field: An Experiment in Public Transportation," Management Science, INFORMS, vol. 64(3), pages 1081-1100, March.
    16. Pascal Achard & Sigrid Suetens, 2023. "The causal effect of ethnic diversity on support for redistribution and the role of discrimination," Journal of Economic Surveys, Wiley Blackwell, vol. 37(5), pages 1678-1696, December.
    17. Kristina M. Bott & Alexander W. Cappelen & Erik Ø. Sørensen & Bertil Tungodden, 2020. "You’ve Got Mail: A Randomized Field Experiment on Tax Evasion," Management Science, INFORMS, vol. 66(7), pages 2801-2819, July.
    18. Levitt, Steven D. & List, John A., 2009. "Field experiments in economics: The past, the present, and the future," European Economic Review, Elsevier, vol. 53(1), pages 1-18, January.
    19. Gravert, Christina, 2013. "How luck and performance affect stealing," Journal of Economic Behavior & Organization, Elsevier, vol. 93(C), pages 301-304.
    20. Pol Campos-Mercade & Claes Ek & Magnus Soederberg & Florian H. Schneider, 2025. "Social Preferences and Environmental Externalities," CEBI working paper series 25-06, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • C99 - Mathematical and Quantitative Methods - - Design of Experiments - - - Other

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00804. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Franco Daniel Albino (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.