IDEAS home Printed from https://ideas.repec.org/a/ucp/jlabec/doi10.1086-671797.html
   My bibliography  Save this article

Evaluating Education Programs That Have Lotteried Admission and Selective Attrition

Author

Listed:
  • John Engberg
  • Dennis Epple
  • Jason Imbrogno
  • Holger Sieg
  • Ron Zimmer

Abstract

We study the effectiveness of magnet programs in an urban district that ration excess demand by admission lotteries. Differential attrition arises since students who lose the lottery are more likely to pursue options outside the school district than students who win the lottery. When students leave the district, important outcome variables are often not observed. The treatment effects are not point-identified. We exploit known quantiles of the outcome distribution to construct informative bounds on treatment effects. We find that magnet programs improve behavioral outcomes but have no significant effect on achievement.

Suggested Citation

  • John Engberg & Dennis Epple & Jason Imbrogno & Holger Sieg & Ron Zimmer, 2014. "Evaluating Education Programs That Have Lotteried Admission and Selective Attrition," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 27-63.
    • Handle: RePEc:ucp:jlabec:doi:10.1086/671797
    DOI: 10.1086/671797
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1086/671797
    Download Restriction: Access to the online full text or PDF requires a subscription.
    File URL: http://dx.doi.org/10.1086/671797
    Download Restriction: Access to the online full text or PDF requires a subscription.
    File URL: https://libkey.io/10.1086/671797?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    2. Will Dobbie & Roland G. Fryer, Jr, 2009. "Are High Quality Schools Enough to Close the Achievement Gap? Evidence from a Social Experiment in Harlem," NBER Working Papers 15473, National Bureau of Economic Research, Inc.
    3. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    4. Richard Blundell & Amanda Gosling & Hidehiko Ichimura & Costas Meghir, 2007. "Changes in the Distribution of Male and Female Wages Accounting for Employment Composition Using Bounds," Econometrica, Econometric Society, vol. 75(2), pages 323-363, March.
    5. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    6. Lee, Lung-Fei, 1979. "Identification and Estimation in Binary Choice Models with Limited (Censored) Dependent Variables," Econometrica, Econometric Society, vol. 47(4), pages 977-996, July.
    7. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    8. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    9. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    10. Zhang, Junni L. & Rubin, Donald B. & Mealli, Fabrizia, 2009. "Likelihood-Based Analysis of Causal Effects of Job-Training Programs Using Principal Stratification," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 166-176.
    11. repec:adr:anecst:y:2008:i:91-92:p:12 is not listed on IDEAS
    12. Guido W. Imbens & Charles F. Manski, 2004. "Confidence Intervals for Partially Identified Parameters," Econometrica, Econometric Society, vol. 72(6), pages 1845-1857, November.
    13. Michael Lechner & Markus Froelich, 2010. "Combining Matching and Nonparametric IV Estimation: Theory and an Application to the Evaluation of Active Labour Market Policies," University of St. Gallen Department of Economics working paper series 2010 2010-21, Department of Economics, University of St. Gallen.
    14. Scott A. Imberman, 2011. "Achievement and Behavior in Charter Schools: Drawing a More Complete Picture," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 416-435, May.
    15. Caroline M. Hoxby & Sonali Murarka, 2009. "Charter Schools in New York City: Who Enrolls and How They Affect Their Students' Achievement," NBER Working Papers 14852, National Bureau of Economic Research, Inc.
    16. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    17. Constantine E. Frangakis & Donald B. Rubin, 2002. "Principal Stratification in Causal Inference," Biometrics, The International Biometric Society, vol. 58(1), pages 21-29, March.
    18. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    19. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    20. Victor Chernozhukov & Han Hong & Elie Tamer, 2007. "Estimation and Confidence Regions for Parameter Sets in Econometric Models," Econometrica, Econometric Society, vol. 75(5), pages 1243-1284, September.
    21. Mitali Das & Whitney K. Newey & Francis Vella, 2003. "Nonparametric Estimation of Sample Selection Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 70(1), pages 33-58.
    22. Barnard J. & Frangakis C.E. & Hill J.L. & Rubin D.B., 2003. "Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 299-323, January.
    23. Ahn, Hyungtaik & Powell, James L., 1993. "Semiparametric estimation of censored selection models with a nonparametric selection mechanism," Journal of Econometrics, Elsevier, vol. 58(1-2), pages 3-29, July.
    24. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    25. Cecilia Elena Rouse, 1998. "Private School Vouchers and Student Achievement: An Evaluation of the Milwaukee Parental Choice Program," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 113(2), pages 553-602.
    26. Heckman, James J, 1974. "Shadow Prices, Market Wages, and Labor Supply," Econometrica, Econometric Society, vol. 42(4), pages 679-694, July.
    27. Robert Moffitt, 2008. "Estimating Marginal Treatment Effects in Heterogeneous Populations," Annals of Economics and Statistics, GENES, issue 91-92, pages 239-261.
    28. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    29. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    2. Sa A. Bui & Steven G. Craig & Scott A. Imberman, 2011. "Is Gifted Education a Bright Idea? Assessing the Impact of Gifted and Talented Programs on Achievement," NBER Working Papers 17089, National Bureau of Economic Research, Inc.
    3. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    4. Patrick Kline & Christopher R. Walters, 2016. "Evaluating Public Programs with Close Substitutes: The Case of HeadStart," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1795-1848.
    5. Atila Abdulkadiroğlu & Nikhil Agarwal & Parag A. Pathak, 2015. "The Welfare Effects of Coordinated Assignment: Evidence from the NYC HS Match," NBER Working Papers 21046, National Bureau of Economic Research, Inc.
    6. Andrew Bibler & Stephen B. Billings, 2020. "Win or Lose: Residential Sorting After a School Choice Lottery," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 457-472, July.
    7. Michael Lechner & Blaise Melly, 2010. "Partial Idendification of Wage Effects of Training Programs," Working Papers 2010-8, Brown University, Department of Economics.
    8. Cook, Jason B., 2018. "Race-Blind Admissions, School Segregation, and Student Outcomes: Evidence from Race-Blind Magnet School Lotteries," IZA Discussion Papers 11909, Institute of Labor Economics (IZA).
    9. Dennis Epple & Richard Romano & Ron Zimmer, 2015. "Charter Schools: A Survey of Research on Their Characteristics and Effectiveness," NBER Working Papers 21256, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John Engberg & Dennis Epple & Jason Imbrogno & Holger Sieg & Ron Zimmer, 2009. "Estimation of Causal Effects in Experiments with Multiple Sources of Noncompliance," NBER Working Papers 14842, National Bureau of Economic Research, Inc.
    2. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    3. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    4. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    5. Martin Huber, 2014. "Treatment Evaluation in the Presence of Sample Selection," Econometric Reviews, Taylor & Francis Journals, vol. 33(8), pages 869-905, November.
    6. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    7. Giovanni Mellace & Roberto Rocci, 2011. "Principal Stratification in sample selection problems with non normal error terms," CEIS Research Paper 194, Tor Vergata University, CEIS, revised 02 May 2011.
    8. Martin Huber & Giovanni Mellace, 2014. "Testing exclusion restrictions and additive separability in sample selection models," Empirical Economics, Springer, vol. 47(1), pages 75-92, August.
    9. Matthias Westphal & Daniel A Kamhöfer & Hendrik Schmitz, 2022. "Marginal College Wage Premiums Under Selection Into Employment," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2231-2272.
    10. Cornelissen, Thomas & Dustmann, Christian & Raute, Anna & Schönberg, Uta, 2016. "From LATE to MTE: Alternative methods for the evaluation of policy interventions," Labour Economics, Elsevier, vol. 41(C), pages 47-60.
    11. Michael Lechner & Blaise Melly, 2007. "Earnings Effects of Training Programs," University of St. Gallen Department of Economics working paper series 2007 2007-28, Department of Economics, University of St. Gallen.
    12. Lina Zhang & David T. Frazier & D. S. Poskitt & Xueyan Zhao, 2020. "Decomposing Identification Gains and Evaluating Instrument Identification Power for Partially Identified Average Treatment Effects," Papers 2009.02642, arXiv.org, revised Sep 2022.
    13. Michael Lechner & Blaise Melly, 2010. "Partial Idendification of Wage Effects of Training Programs," Working Papers 2010-8, Brown University, Department of Economics.
    14. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    15. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    16. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    17. Xintong Wang & Carlos A. Flores & Alfonso Flores-Lagunes, 2020. "The Effects of Vietnam-Era Military Service on the Long-Term Health of Veterans: A Bounds Analysis," Center for Policy Research Working Papers 234, Center for Policy Research, Maxwell School, Syracuse University.
    18. Baldwin, Kate & Bhavnani, Rikhil R., 2013. "Ancillary Experiments: Opportunities and Challenges," WIDER Working Paper Series 024, World Institute for Development Economic Research (UNU-WIDER).
    19. Markus Frölich & Martin Huber, 2014. "Treatment Evaluation With Multiple Outcome Periods Under Endogeneity and Attrition," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1697-1711, December.
    20. Robert Moffitt, 1991. "Program Evaluation With Nonexperimental Data," Evaluation Review, , vol. 15(3), pages 291-314, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ucp:jlabec:doi:10.1086/671797. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Journals Division (email available below). General contact details of provider: https://www.journals.uchicago.edu/JOLE .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.