IDEAS home Printed from https://ideas.repec.org/p/hal/journl/halshs-02973595.html
   My bibliography  Save this paper

Estimating the Effect of Treatments Allocated by Randomized Waiting Lists

Author

Listed:
  • Clément Chaisemartin

    (UCSB - University of California [Santa Barbara] - University of California)

  • Luc Behaghel

    (PSE - Paris School of Economics - ENPC - École des Ponts ParisTech - ENS Paris - École normale supérieure - Paris - PSL - Université Paris sciences et lettres - UP1 - Université Paris 1 Panthéon-Sorbonne - CNRS - Centre National de la Recherche Scientifique - EHESS - École des hautes études en sciences sociales - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement, PJSE - Paris Jourdan Sciences Economiques - UP1 - Université Paris 1 Panthéon-Sorbonne - ENS Paris - École normale supérieure - Paris - PSL - Université Paris sciences et lettres - EHESS - École des hautes études en sciences sociales - ENPC - École des Ponts ParisTech - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement)

Abstract

Oversubscribed treatments are often allocated using randomized waiting lists. Applicants are ranked randomly, and treatment offers are made following that ranking until all seats are filled. To estimate causal effects, researchers often compare applicants getting and not getting an offer. We show that those two groups are not statistically comparable. Therefore, the estimator arising from that comparison is inconsistent when the number of waitlists goes to infinity. We propose a new estimator, and show that it is consistent, provided the waitlists have at least two seats. Finally, we revisit an application, and we show that using our estimator can lead to a statistically significant difference with respect to the results obtained using the commonly used estimator.

Suggested Citation

  • Clément Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Post-Print halshs-02973595, HAL.
  • Handle: RePEc:hal:journl:halshs-02973595
    DOI: 10.3982/ECTA16032
    Note: View the original document on HAL open archive server: https://halshs.archives-ouvertes.fr/halshs-02973595
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. repec:mpr:mprres:7273 is not listed on IDEAS
    2. Alberto Abadie & Joshua Angrist & Guido Imbens, 2002. "Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings," Econometrica, Econometric Society, vol. 70(1), pages 91-117, January.
    3. Paloma Acevedo & Guillermo Cruces & Paul Gertler & Sebastian Martinez, 2017. "Living Up to Expectations: How Job Training Made Women Better Off and Men Worse Off," NBER Working Papers 23264, National Bureau of Economic Research, Inc.
    4. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    5. Vilsa E. Curto & Roland G. Fryer Jr., 2014. "The Potential of Urban Boarding Schools for the Poor: Evidence from SEED," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 65-93.
    6. Sarojini Hirshleifer & David McKenzie & Rita Almeida & Cristobal Ridao‐Cano, 2016. "The Impact of Vocational Training for the Unemployed: Experimental Evidence from Turkey," Economic Journal, Royal Economic Society, vol. 126(597), pages 2115-2146, November.
    7. Andrew V. Carter & Kevin T. Schnepel & Douglas G. Steigerwald, 2017. "Asymptotic Behavior of a t -Test Robust to Cluster Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 698-709, July.
    8. David J. Deming, 2014. "Using School Choice Lotteries to Test Measures of School Effectiveness," American Economic Review, American Economic Association, vol. 104(5), pages 406-411, May.
    9. Bruno Crépon & Florencia Devoto & Esther Duflo & William Parienté, 2015. "Estimating the Impact of Microcredit on Those Who Take It Up: Evidence from a Randomized Experiment in Morocco," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 123-150, January.
    10. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2012. "The Effect of School Choice on Intrinsic Motivation and Academic Outcomes," NBER Working Papers 18324, National Bureau of Economic Research, Inc.
    11. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    12. David Card & Pablo Ibarrarán & Ferdinando Regalia & David Rosas-Shady & Yuri Soares, 2011. "The Labor Market Impacts of Youth Training in the Dominican Republic," Journal of Labor Economics, University of Chicago Press, vol. 29(2), pages 267-300.
    13. Luc Behaghel & Clément de Chaisemartin & Marc Gurgand, 2017. "Ready for Boarding? The Effects of a Boarding School for Disadvantaged Students," American Economic Journal: Applied Economics, American Economic Association, vol. 9(1), pages 140-164, January.
    14. de Chaisemartin, Clement, 2013. "Defying the LATE? Identification of local treatment effects when the instrument violates monotonicity," The Warwick Economics Research Paper Series (TWERPS) 1020, University of Warwick, Department of Economics.
    15. Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
    16. de Chaisemartin, Clement, 2013. "Defying the LATE? Identication of local treatment eects when the instrument violates monotonicity," Economic Research Papers 270439, University of Warwick - Department of Economics.
    17. Orazio Attanasio & Adriana Kugler & Costas Meghir, 2011. "Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 188-220, July.
    18. Melissa A. Clark & Philip Gleason & Christina Clark Tuttle & Marsha K. Silverberg, 2011. "Do Charter Schools Improve Student Achievement? Evidence from a National Randomized Study," Mathematica Policy Research Reports af41392138504f369930e6f2b, Mathematica Policy Research.
    19. Clément de Chaisemartin, 2017. "Tolerating defiance? Local average treatment effects without monotonicity," Quantitative Economics, Econometric Society, vol. 8(2), pages 367-396, July.
    20. Gut, Allan, 1992. "The weak law of large numbers for arrays," Statistics & Probability Letters, Elsevier, vol. 14(1), pages 49-52, May.
    21. Clément de Chaisemartin & Xavier d'Haultfoeuille, 2012. "Late Again with Defiers," PSE Working Papers halshs-00699646, HAL.
    22. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, Oxford University Press, vol. 132(2), pages 871-919.
    23. Atila Abdulkadiroğlu & Joshua D. Angrist & Peter D. Hull & Parag A. Pathak, 2016. "Charters without Lotteries: Testing Takeovers in New Orleans and Boston," American Economic Review, American Economic Association, vol. 106(7), pages 1878-1920, July.
    24. Atila Abdulkadiroğlu & Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak, 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots," The Quarterly Journal of Economics, Oxford University Press, vol. 126(2), pages 699-748.
    25. Blattman, Christopher & Annan, Jeannie, 2016. "Can Employment Reduce Lawlessness and Rebellion? A Field Experiment with High-Risk Men in a Fragile State," American Political Science Review, Cambridge University Press, vol. 110(1), pages 1-17, February.
    26. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, August.
    27. Caroline M. Hoxby & Sonali Murarka, 2009. "Charter Schools in New York City: Who Enrolls and How They Affect Their Students' Achievement," NBER Working Papers 14852, National Bureau of Economic Research, Inc.
    28. Matthew A. Kraft, 2014. "How to Make Additional Time Matter: Integrating Individualized Tutorials into an Extended Day," Education Finance and Policy, MIT Press, vol. 10(1), pages 81-116, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Araya, Federico & Le Barbanchon, Thomas & Ubfal, Diego, 2019. "The Effects of Working while in School: Evidence from Uruguayan Lotteries," CEPR Discussion Papers 13826, C.E.P.R. Discussion Papers.
    2. Daniel Da Mata & Rodrigo C. Oliveira & Diana Silva, 2021. "Who benefits from job training programmes?: Evidence from a high-dosage programme in Brazil," WIDER Working Paper Series wp-2021-121, World Institute for Development Economic Research (UNU-WIDER).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kaspar Wüthrich, 2020. "A Comparison of Two Quantile Models With Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 443-456, April.
    2. Song, Yang, 2019. "Sorting, school performance and quality: Evidence from China," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 238-261.
    3. Luc Behaghel & Clément de Chaisemartin & Marc Gurgand, 2017. "Ready for Boarding? The Effects of a Boarding School for Disadvantaged Students," American Economic Journal: Applied Economics, American Economic Association, vol. 9(1), pages 140-164, January.
    4. Clément de Chaisemartin & Xavier d'Haultfoeuille, 2012. "Late Again with Defiers," Working Papers halshs-00699646, HAL.
    5. Johanna Fajardo-Gonzalez, 2021. "Domestic violence, decision-making power, and female employment in Colombia," Review of Economics of the Household, Springer, vol. 19(1), pages 233-254, March.
    6. Caterina Calsamiglia & Maia Guell, 2014. "The Illusion of School Choice: Empirical Evidence from Barcelona," Working Papers 712, Federal Reserve Bank of Minneapolis.
    7. de Chaisemartin, Clement, 2013. "Defying the LATE? Identication of local treatment eects when the instrument violates monotonicity," Economic Research Papers 270439, University of Warwick - Department of Economics.
    8. Éric Gautier, 2021. "Relaxing Monotonicity in Endogenous Selection Models and Application to Surveys," Post-Print hal-03306234, HAL.
    9. de Chaisemartin, Clement, 2013. "Defying the LATE? Identification of local treatment effects when the instrument violates monotonicity," The Warwick Economics Research Paper Series (TWERPS) 1020, University of Warwick, Department of Economics.
    10. Chakravarty, Shubha & Lundberg, Mattias & Nikolov, Plamen & Zenker, Juliane, 2019. "Vocational training programs and youth labor market outcomes: Evidence from Nepal," Journal of Development Economics, Elsevier, vol. 136(C), pages 71-110.
    11. Huntington-Klein Nick, 2020. "Instruments with Heterogeneous Effects: Bias, Monotonicity, and Localness," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 182-208, January.
    12. Clément de Chaisemartin, 2017. "Tolerating defiance? Local average treatment effects without monotonicity," Quantitative Economics, Econometric Society, vol. 8(2), pages 367-396, July.
    13. Schmieder, Julia, 2020. "Fertility as a Driver of Maternal Employment," IZA Discussion Papers 13496, Institute of Labor Economics (IZA).
    14. Atila Abdulkadiroğlu & Parag A. Pathak & Christopher R. Walters, 2018. "Free to Choose: Can School Choice Reduce Student Achievement?," American Economic Journal: Applied Economics, American Economic Association, vol. 10(1), pages 175-206, January.
    15. Huntington-Klein Nick, 2020. "Instruments with Heterogeneous Effects: Bias, Monotonicity, and Localness," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 182-208, January.
    16. Julia Schmieder, 2020. "Fertility as a Driver of Maternal Employment," Discussion Papers of DIW Berlin 1882, DIW Berlin, German Institute for Economic Research.
    17. Brigham R. Frandsen & Lars J. Lefgren & Emily C. Leslie, 2019. "Judging Judge Fixed Effects," NBER Working Papers 25528, National Bureau of Economic Research, Inc.
    18. Eyles, Andrew & Machin, Stephen & McNally, Sandra, 2017. "Unexpected school reform: Academisation of primary schools in England," Journal of Public Economics, Elsevier, vol. 155(C), pages 108-121.
    19. Carlson, Deven & Lavertu, Stéphane, 2016. "Charter school closure and student achievement: Evidence from Ohio," Journal of Urban Economics, Elsevier, vol. 95(C), pages 31-48.
    20. Margaret Brehm & Scott A. Imberman & Michael Naretta, 2017. "Capitalization of Charter Schools into Residential Property Values," Education Finance and Policy, MIT Press, vol. 12(1), pages 1-27, Winter.

    More about this item

    Keywords

    Waiting lists; Non-takers; Non compliance; Instrumental variable; Local average treatment effect; Randomized controlled trials;
    All these keywords.

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:halshs-02973595. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://hal.archives-ouvertes.fr/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.