IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v31y2012i2p237-253.html
   My bibliography  Save this article

Using lotteries to evaluate schools of choice: Evidence from a national study of charter schools

Author

Listed:
  • Tuttle, Christina Clark
  • Gleason, Philip
  • Clark, Melissa

Abstract

This paper draws on data and experiences observing and analyzing school lotteries from the National Evaluation of Charter School Impacts (Gleason et al., 2010) to describe the challenges associated with lottery-based research. In that study, covering 36 charter middle schools in 15 states, we found that charter schools did not affect student achievement or behavior on average, although there was substantial variation across schools. In this paper, we discuss the prevalence of oversubscribed charter schools at the time the study was conducted (the 2005–2006 and 2006–2007 school years), which was lower than commonly reported. We then describe how the sample of schools that participated in the study compared to all other charter middle schools nationwide, to provide some insight into the generalizabilty of findings from lottery-based studies. In general, oversubscribed charter schools were more likely to be located in urban areas and serve a higher-achieving population of students than those without excess demand. We also describe common features of school lotteries and waitlists, and examine implications of these features for a school's ability to support a lottery-based study. Finally, we summarize lessons learned for conducting lottery-based research on charter schools, drawing on our observations of the schools’ lotteries and analysis of the data from these lotteries.

Suggested Citation

  • Tuttle, Christina Clark & Gleason, Philip & Clark, Melissa, 2012. "Using lotteries to evaluate schools of choice: Evidence from a national study of charter schools," Economics of Education Review, Elsevier, vol. 31(2), pages 237-253.
  • Handle: RePEc:eee:ecoedu:v:31:y:2012:i:2:p:237-253
    DOI: 10.1016/j.econedurev.2011.07.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775711001051
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.econedurev.2011.07.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak & Christopher R. Walters, 2012. "Who Benefits from KIPP?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 31(4), pages 837-860, September.
    2. repec:mpr:mprres:6676 is not listed on IDEAS
    3. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    4. Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
    5. Dee, Thomas S. & Fu, Helen, 2004. "Do charter schools skim students or drain resources?," Economics of Education Review, Elsevier, vol. 23(3), pages 259-271, June.
    6. repec:mpr:mprres:7293 is not listed on IDEAS
    7. repec:mpr:mprres:6720 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zimmer, Ron & Gill, Brian & Booker, Kevin & Lavertu, Stéphane & Witte, John, 2012. "Examining charter student achievement effects across seven states," Economics of Education Review, Elsevier, vol. 31(2), pages 213-224.
    2. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    3. Cid, Alejandro & Balsa, Ana, 2012. "Impact Evaluation of a Privately Managed Tuition-Free Middle school in a Poor Neighborhood in Montevideo," MPRA Paper 39913, University Library of Munich, Germany.
    4. repec:mpr:mprres:8005 is not listed on IDEAS
    5. Sorensen, Lucy C. & Holt, Stephen B., 2021. "Sorting it Out: The Effects of Charter Expansion on Teacher and Student Composition at Traditional Public Schools," Economics of Education Review, Elsevier, vol. 82(C).
    6. Kerry A. Adzima, 2017. "College Readiness: Are California Charter School Students Better Prepared?," Economic Affairs, Wiley Blackwell, vol. 37(2), pages 182-196, June.
    7. Matthew Davis & Blake Heller, 2019. "No Excuses Charter Schools and College Enrollment: New Evidence from a High School Network in Chicago," Education Finance and Policy, MIT Press, vol. 14(3), pages 414-440, Summer.
    8. Kevin Booker & Brian Gill & Tim Sass & Ron Zimmer, "undated". "Charter High Schools' Effects on Long-Term Attainment and Earnings (Working Paper)," Mathematica Policy Research Reports cfe561a4b1924b7eafb64f918, Mathematica Policy Research.
    9. Toma, Eugenia & Zimmer, Ron, 2012. "Two decades of charter schools: Expectations, reality, and the future," Economics of Education Review, Elsevier, vol. 31(2), pages 209-212.
    10. Ni, Yongmei & Rorrer, Andrea K., 2012. "Twice considered: Charter schools and student achievement in Utah," Economics of Education Review, Elsevier, vol. 31(5), pages 835-849.
    11. Andrew Bibler & Stephen B. Billings, 2020. "Win or Lose: Residential Sorting After a School Choice Lottery," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 457-472, July.
    12. repec:mpr:mprres:7927 is not listed on IDEAS
    13. Diana McCallum & Christina Tuttle & Jeffrey Max & Brian Gill & Philip Gleason, "undated". "How Does School Choice Affect Racial Integration?," Mathematica Policy Research Reports 60be07fb436f46b6b40a9178f, Mathematica Policy Research.
    14. Alexandra Resch & Jillian Berk & Lauren Akers, "undated". "Recognizing and Conducting Opportunistic Experiments in Education: A Guide for Policymakers and Researchers," Mathematica Policy Research Reports b58a999ab27a4cabafac5aa08, Mathematica Policy Research.
    15. repec:mpr:mprres:8126 is not listed on IDEAS
    16. Davis, Devora H. & Raymond, Margaret E., 2012. "Choices for studying choice: Assessing charter school effectiveness using two quasi-experimental methods," Economics of Education Review, Elsevier, vol. 31(2), pages 225-236.
    17. Carruthers, Celeste K., 2012. "New schools, new students, new teachers: Evaluating the effectiveness of charter schools," Economics of Education Review, Elsevier, vol. 31(2), pages 280-292.
    18. Brian Gill & Joshua Furgeson & Hanley S. Chiang & Bing-Ru Teh & Joshua Haimson & Natalya Verbitsky-Savitz, "undated". "Replicating Experimental Impact Estimates with Nonexperimental Methods in the Context of Control Crossover," Mathematica Policy Research Reports 2798055510274fa9b4fdfa54b, Mathematica Policy Research.
    19. Dennis Epple & Richard Romano & Ron Zimmer, 2015. "Charter Schools: A Survey of Research on Their Characteristics and Effectiveness," NBER Working Papers 21256, National Bureau of Economic Research, Inc.
    20. Singleton, John D., 2017. "Putting dollars before scholars? Evidence from for-profit charter schools in Florida," Economics of Education Review, Elsevier, vol. 58(C), pages 43-54.
    21. Alicia Meckstroth & Alexandra Resch & Jonathan McCay & Michelle Derr & Jillian Berk & Lauren Akers, "undated". "Advancing Evidence-Based Decision Making: A Toolkit on Recognizing and Conducting Opportunistic Experiments in the Family Self-Sufficiency and Stability Policy Area," Mathematica Policy Research Reports 1b4e98bbc73c4b6a80f6ba1c7, Mathematica Policy Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roland G. Fryer, Jr, 2016. "The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments," NBER Working Papers 22130, National Bureau of Economic Research, Inc.
    2. Atila Abdulkadiroğlu & Joshua Angrist & Parag Pathak, 2014. "The Elite Illusion: Achievement Effects at Boston and New York Exam Schools," Econometrica, Econometric Society, vol. 82(1), pages 137-196, January.
    3. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    4. Song, Yang, 2019. "Sorting, school performance and quality: Evidence from China," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 238-261.
    5. Sorensen, Lucy C. & Holt, Stephen B., 2021. "Sorting it Out: The Effects of Charter Expansion on Teacher and Student Composition at Traditional Public Schools," Economics of Education Review, Elsevier, vol. 82(C).
    6. Neilson, Christopher A. & Zimmerman, Seth D., 2014. "The effect of school construction on test scores, school enrollment, and home prices," Journal of Public Economics, Elsevier, vol. 120(C), pages 18-31.
    7. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    8. Carlson, Deven & Lavertu, Stéphane, 2016. "Charter school closure and student achievement: Evidence from Ohio," Journal of Urban Economics, Elsevier, vol. 95(C), pages 31-48.
    9. Facundo Albornoz & Samuel Berlinski & Antonio Cabrales, 2016. "Motivation, Resources and the Organization of the School System," IDB Publications (Working Papers) 94958, Inter-American Development Bank.
    10. Margaret Brehm & Scott A. Imberman & Michael Naretta, 2017. "Capitalization of Charter Schools into Residential Property Values," Education Finance and Policy, MIT Press, vol. 12(1), pages 1-27, Winter.
    11. Vilsa E. Curto & Roland G. Fryer Jr., 2014. "The Potential of Urban Boarding Schools for the Poor: Evidence from SEED," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 65-93.
    12. Julie Berry Cullen & Steven D. Levitt & Erin Robertson & Sally Sadoff, 2013. "What Can Be Done to Improve Struggling High Schools?," Journal of Economic Perspectives, American Economic Association, vol. 27(2), pages 133-152, Spring.
    13. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2012. "The Effect of School Choice on Intrinsic Motivation and Academic Outcomes," Working Papers 2012-3, Princeton University. Economics Department..
    14. Richard Murphy & Felix Weinhardt, 2020. "Top of the Class: The Importance of Ordinal Rank," Review of Economic Studies, Oxford University Press, vol. 87(6), pages 2777-2826.
    15. Ha, Wei & Kang, Le & Song, Yang, 2020. "College matching mechanisms and matching stability: Evidence from a natural experiment in China," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 206-226.
    16. Nicholas Bloom & Renata Lemos & Raffaella Sadun & John Van Reenen, 2015. "Does Management Matter in schools?," Economic Journal, Royal Economic Society, vol. 0(584), pages 647-674, May.
    17. Nirav Mehta, 2017. "Competition In Public School Districts: Charter School Entry, Student Sorting, And School Input Determination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 58(4), pages 1089-1116, November.
    18. Laura Grube & Devin Anderson, 2018. "School Choice and Charter Schools in Review: What Have We Learned?," Journal of Private Enterprise, The Association of Private Enterprise Education, vol. 33(Winter 20), pages 21-44.
    19. Calsamiglia, Caterina & Güell, Maia, 2014. "The Illusion of School Choice: Empirical Evidence from Barcelona," CEPR Discussion Papers 10011, C.E.P.R. Discussion Papers.
    20. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.

    More about this item

    Keywords

    School choice; Charter schools; Lotteries; Randomized control trials;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:31:y:2012:i:2:p:237-253. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/econedurev .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.