IDEAS home Printed from https://ideas.repec.org/p/eid/wpaper/39843.html
   My bibliography  Save this paper

A New Perspective on the Issue of Selection Bias into Randomized Controlled Field Experiments

Author

Listed:
  • Michele Belot

    (University of Edinburgh)

  • Jonathan James

    (University of Bath)

Abstract

Many randomized controlled trials require participants to opt in. Such self-selection could introduce a potential bias, because only the most optimistic may participate. We revisit this prediction. We argue that in many situations, the experimental intervention is competing with alternative interventions participants could conduct themselves outside the experiment. Since participants have a chance of being assigned to the control group, participating has a direct opportunity cost, which is likely to be higher for optimists. We propose a model of self-selection and show that both pessimists and optimists may opt out of the experiment, leading to an ambiguous selection bias.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Michele Belot & Jonathan James, 2014. "A New Perspective on the Issue of Selection Bias into Randomized Controlled Field Experiments," Department of Economics Working Papers 23/14, University of Bath, Department of Economics.
  • Handle: RePEc:eid:wpaper:39843
    as

    Download full text from publisher

    File URL: https://purehost.bath.ac.uk/ws/files/62506384/23_14.pdf
    File Function: Final published version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 531-580.
    2. Roland G. Fryer, 2011. "Financial Incentives and Student Achievement: Evidence from Randomized Trials," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1755-1798.
    3. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2013. "Team Incentives: Evidence From A Firm Level Experiment," Journal of the European Economic Association, European Economic Association, vol. 11(5), pages 1079-1114, October.
    4. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    5. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
    6. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    7. Edward Vytlacil & James J. Heckman, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May.
    8. Justine S. Hastings & Jeffrey M. Weinstein, 2008. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments," The Quarterly Journal of Economics, Oxford University Press, vol. 123(4), pages 1373-1414.
    9. List, John A. & Rasul, Imran, 2011. "Field Experiments in Labor Economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 2, pages 103-228, Elsevier.
    10. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2007. "Incentives for Managers and Inequality among Workers: Evidence from a Firm-Level Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 122(2), pages 729-773.
    11. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    12. Jessica Wisdom & Julie S. Downs & George Loewenstein, 2010. "Promoting Healthy Choices: Information versus Convenience," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 164-178, April.
    13. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    14. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    15. Justine S. Hastings & Jeffrey M. Weinstein, 2008. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments," The Quarterly Journal of Economics, Oxford University Press, vol. 123(4), pages 1373-1414.
    16. Malani, Anup, 2008. "Patient enrollment in medical trials: Selection bias in a randomized experiment," Journal of Econometrics, Elsevier, vol. 144(2), pages 341-351, June.
    17. Pieter A. Gautier & Bas van der Klaauw, 2012. "Selection in a field experiment with voluntary participation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(1), pages 63-84, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    2. Naphtal Habiyaremye & Nadhem Mtimet & Emily A. Ouma & Gideon A. Obare, 2023. "Consumers' willingness to pay for safe and quality milk: Evidence from experimental auctions in Rwanda," Agribusiness, John Wiley & Sons, Ltd., vol. 39(4), pages 1049-1074, October.
    3. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    4. Michael Sanders & Aisling Ní Chonaire, 2015. "“Powered to Detect Small Effect Sizes”: You keep saying that. I do not think it means what you think it means," The Centre for Market and Public Organisation 15/337, The Centre for Market and Public Organisation, University of Bristol, UK.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    2. Altmann, Steffen & Falk, Armin & Jäger, Simon & Zimmermann, Florian, 2018. "Learning about job search: A field experiment with job seekers in Germany," Journal of Public Economics, Elsevier, vol. 164(C), pages 33-49.
    3. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    4. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    5. Brigitte C. Madrian, 2014. "Applying Insights from Behavioral Economics to Policy Design," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 663-688, August.
    6. Hunt Allcott & Richard L. Sweeney, 2017. "The Role of Sales Agents in Information Disclosure: Evidence from a Field Experiment," Management Science, INFORMS, vol. 63(1), pages 21-39, January.
    7. Prina, Silvia & Royer, Heather, 2014. "The importance of parental knowledge: Evidence from weight report cards in Mexico," Journal of Health Economics, Elsevier, vol. 37(C), pages 232-247.
    8. Silvia Prina & Heather Royer, 2013. "The Importance of Parental Knowledge and Social Norms: Evidence from Weight Report Cards in Mexico," NBER Working Papers 19344, National Bureau of Economic Research, Inc.
    9. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    10. Jensen, Robert & Lleras-Muney, Adriana, 2012. "Does staying in school (and not working) prevent teen smoking and drinking?," Journal of Health Economics, Elsevier, vol. 31(4), pages 644-657.
    11. Nano Barahona & Cristóbal Otero & Sebastián Otero, 2023. "Equilibrium Effects of Food Labeling Policies," Econometrica, Econometric Society, vol. 91(3), pages 839-868, May.
    12. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank Group, vol. 33(1), pages 34-64.
    13. Steven D. Levitt & John A. List & Susanne Neckermann & Sally Sadoff, 2016. "The Behavioralist Goes to School: Leveraging Behavioral Economics to Improve Educational Performance," American Economic Journal: Economic Policy, American Economic Association, vol. 8(4), pages 183-219, November.
    14. Kamali Shahdadi, Behrang, 2021. "The effects of student composition on teachers' effort and students' performance: Implications for tracking, school choice, and affirmative action," Games and Economic Behavior, Elsevier, vol. 130(C), pages 384-399.
    15. Greer K. Gosnell & John A. List & Robert Metcalfe, 2016. "A New Approach to an Age-Old Problem: Solving Externalities by Incenting Workers Directly," NBER Working Papers 22316, National Bureau of Economic Research, Inc.
    16. Andrew Dustan & Alain de Janvry & Elisabeth Sadoulet, 2017. "Flourish or Fail?: The Risky Reward of Elite High School Admission in Mexico City," Journal of Human Resources, University of Wisconsin Press, vol. 52(3), pages 756-799.
    17. Charness, Gary & Kuhn, Peter, 2011. "Lab Labor: What Can Labor Economists Learn from the Lab?," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 3, pages 229-330, Elsevier.
    18. Zhang, Hongliang, 2016. "Identification of treatment effects under imperfect matching with an application to Chinese elite schools," Journal of Public Economics, Elsevier, vol. 142(C), pages 56-82.
    19. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    20. Linda Babcock & William Congdon & Lawrence Katz & Sendhil Mullainathan, 2012. "Notes on behavioral economics and labor market policy," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 1(1), pages 1-14, December.

    More about this item

    JEL classification:

    • C4 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eid:wpaper:39843. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Scholarly Communications Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/debatuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.