IDEAS home Printed from https://ideas.repec.org/p/eid/wpaper/39843.html
   My bibliography  Save this paper

A New Perspective on the Issue of Selection Bias into Randomized Controlled Field Experiments

Author

Listed:
  • Michele Belot

    (University of Edinburgh)

  • Jonathan James

    (University of Bath)

Abstract

Many randomized controlled trials require participants to opt in. Such self-selection could introduce a potential bias, because only the most optimistic may participate. We revisit this prediction. We argue that in many situations, the experimental intervention is competing with alternative interventions participants could conduct themselves outside the experiment. Since participants have a chance of being assigned to the control group, participating has a direct opportunity cost, which is likely to be higher for optimists. We propose a model of self-selection and show that both pessimists and optimists may opt out of the experiment, leading to an ambiguous selection bias.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Michele Belot & Jonathan James, 2014. "A New Perspective on the Issue of Selection Bias into Randomized Controlled Field Experiments," Department of Economics Working Papers 23/14, University of Bath, Department of Economics.
    • Handle: RePEc:eid:wpaper:39843
    as

    Download full text from publisher

    File URL: https://purehost.bath.ac.uk/ws/files/62506384/23_14.pdf
    File Function: Final published version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
    2. Justine S. Hastings & Jeffrey M. Weinstein, 2008. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(4), pages 1373-1414.
    3. Roland G. Fryer, 2011. "Financial Incentives and Student Achievement: Evidence from Randomized Trials," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1755-1798.
    4. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    5. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2007. "Incentives for Managers and Inequality among Workers: Evidence from a Firm-Level Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(2), pages 729-773.
    6. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2013. "Team Incentives: Evidence From A Firm Level Experiment," Journal of the European Economic Association, European Economic Association, vol. 11(5), pages 1079-1114, October.
    7. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    8. Ernst Fehr & Lorenz Goette, 2007. "Do Workers Work More if Wages Are High? Evidence from a Randomized Field Experiment," American Economic Review, American Economic Association, vol. 97(1), pages 298-317, March.
    9. Malani, Anup, 2008. "Patient enrollment in medical trials: Selection bias in a randomized experiment," Journal of Econometrics, Elsevier, vol. 144(2), pages 341-351, June.
    10. Edward Vytlacil & James J. Heckman, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May.
    11. List, John A. & Rasul, Imran, 2011. "Field Experiments in Labor Economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 2, pages 103-228, Elsevier.
    12. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    13. Pieter A. Gautier & Bas van der Klaauw, 2012. "Selection in a field experiment with voluntary participation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(1), pages 63-84, January.
    14. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    15. Jessica Wisdom & Julie S. Downs & George Loewenstein, 2010. "Promoting Healthy Choices: Information versus Convenience," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 164-178, April.
    16. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    2. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    3. Naphtal Habiyaremye & Nadhem Mtimet & Emily A. Ouma & Gideon A. Obare, 2023. "Consumers' willingness to pay for safe and quality milk: Evidence from experimental auctions in Rwanda," Agribusiness, John Wiley & Sons, Ltd., vol. 39(4), pages 1049-1074, October.
    4. Michael Sanders & Aisling Ní Chonaire, 2015. "“Powered to Detect Small Effect Sizes”: You keep saying that. I do not think it means what you think it means," The Centre for Market and Public Organisation 15/337, The Centre for Market and Public Organisation, University of Bristol, UK.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Belot, Michèle & James, Jonathan, 2016. "Partner selection into policy relevant field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 31-56.
    2. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    3. Altmann, Steffen & Falk, Armin & Jäger, Simon & Zimmermann, Florian, 2018. "Learning about job search: A field experiment with job seekers in Germany," Journal of Public Economics, Elsevier, vol. 164(C), pages 33-49.
    4. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    5. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    6. Gosnell, Greer & Metcalfe, Robert & List, John A, 2016. "A new approach to an age-old problem: solving externalities by incenting workers directly," LSE Research Online Documents on Economics 84331, London School of Economics and Political Science, LSE Library.
    7. Charness, Gary & Kuhn, Peter, 2011. "Lab Labor: What Can Labor Economists Learn from the Lab?," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 3, pages 229-330, Elsevier.
    8. Oriana Bandiera & Iwan Barankay & Imran Rasul, 2011. "Field Experiments with Firms," Journal of Economic Perspectives, American Economic Association, vol. 25(3), pages 63-82, Summer.
    9. Supreet Kaur & Michael Kremer & Sendhil Mullainathan, 2015. "Self-Control at Work," Journal of Political Economy, University of Chicago Press, vol. 123(6), pages 1227-1277.
    10. Brigitte C. Madrian, 2014. "Applying Insights from Behavioral Economics to Policy Design," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 663-688, August.
    11. Charles Bellemare & Steeve Marchand & Bruce Shearer, 2016. "Structural Estimation and Experiments: Applications to Contracting Models," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 172(2), pages 342-363, June.
    12. Hunt Allcott & Richard L. Sweeney, 2017. "The Role of Sales Agents in Information Disclosure: Evidence from a Field Experiment," Management Science, INFORMS, vol. 63(1), pages 21-39, January.
    13. Cai, Xiqian & Jiang, Wei & Song, Hong & Xie, Huihua, 2022. "Pay for performance schemes and manufacturing worker productivity: Evidence from a kinked design in China," Journal of Development Economics, Elsevier, vol. 156(C).
    14. Prina, Silvia & Royer, Heather, 2014. "The importance of parental knowledge: Evidence from weight report cards in Mexico," Journal of Health Economics, Elsevier, vol. 37(C), pages 232-247.
    15. Maibom, Jonas, 2021. "The Danish Labor Market Experiments: Methods and Findings," Nationaløkonomisk tidsskrift, Nationaløkonomisk Forening, vol. 2021(1), pages 1-21.
    16. Silvia Prina & Heather Royer, 2013. "The Importance of Parental Knowledge and Social Norms: Evidence from Weight Report Cards in Mexico," NBER Working Papers 19344, National Bureau of Economic Research, Inc.
    17. David J. Cooper & Krista Saral & Marie Claire Villeval, 2021. "Why Join a Team?," Management Science, INFORMS, vol. 67(11), pages 6980-6997, November.
    18. Guido Friebel & Matthias Heinz & Miriam Krueger & Nikolay Zubanov, 2017. "Team Incentives and Performance: Evidence from a Retail Chain," American Economic Review, American Economic Association, vol. 107(8), pages 2168-2203, August.
    19. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    20. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C4 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eid:wpaper:39843. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Scholarly Communications Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/debatuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.