IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/21705.html
   My bibliography  Save this paper

Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation

Author

Listed:
  • Atila Abdulkadiroglu
  • Joshua D. Angrist
  • Yusuke Narita
  • Parag A. Pathak

Abstract

A growing number of school districts use centralized assignment mechanisms to allocate school seats in a manner that reflects student preferences and school priorities. Many of these assignment schemes use lotteries to ration seats when schools are oversubscribed. The resulting random assignment opens the door to credible quasi-experimental research designs for the evaluation of school effectiveness. Yet the question of how best to separate the lottery-generated variation integral to such designs from non-random preferences and priorities remains open. This paper develops easily-implemented empirical strategies that fully exploit the random assignment embedded in the widely-used deferred acceptance mechanism and its variants. We use these methods to evaluate charter schools in Denver, one of a growing number of districts that integrate charter and traditional public schools in a unified assignment system. The resulting estimates show large achievement gains from charter school attendance. Our approach expands the scope for impact evaluation by maximizing the number of students and schools that can be studied using random assignment. We also show how to use DA to identify causal effects in models with multiple school sectors.

Suggested Citation

  • Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2015. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," NBER Working Papers 21705, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:21705
    Note: ED LS PE TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w21705.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Parag A. Pathak & Christopher R. Walters, 2013. "Explaining Charter School Effectiveness," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 1-27, October.
    2. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    3. Atila Abdulkadiro?lu & Yeon-Koo Che & Yosuke Yasuda, 2015. "Expanding "Choice" in School Choice," American Economic Journal: Microeconomics, American Economic Association, vol. 7(1), pages 1-42, February.
    4. Luc Behaghel & Bruno Crépon & Marc Gurgand, 2013. "Robustness of the encouragement design in a two-treatment randomized control trial," Working Papers halshs-00834169, HAL.
    5. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    6. Atila Abdulkadroǧlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak & Roman A. Zarate, 2017. "Regression Discontinuity in Serial Dictatorship: Achievement Effects at Chicago's Exam Schools," American Economic Review, American Economic Association, vol. 107(5), pages 240-245, May.
    7. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    8. Monique De Haan & Pieter A. Gautier & Hessel Oosterbeek & Bas van der Klaauw, 2023. "The Performance of School Assignment Mechanisms in Practice," Journal of Political Economy, University of Chicago Press, vol. 131(2), pages 388-455.
    9. Caterina Calsamiglia & Chao Fu & Maia Güell, 2014. "Structural Estimation of a Model of School Choices: the Boston Mechanism vs. Its Alternatives," Working Papers 811, Barcelona School of Economics.
    10. Frolich, Markus, 2007. "Nonparametric IV estimation of local average treatment effects with covariates," Journal of Econometrics, Elsevier, vol. 139(1), pages 35-75, July.
    11. , A. & ,, 2011. "Lotteries in student assignment: An equivalence result," Theoretical Economics, Econometric Society, vol. 6(1), January.
    12. Atila Abdulkadiroğlu & Joshua Angrist & Parag Pathak, 2014. "The Elite Illusion: Achievement Effects at Boston and New York Exam Schools," Econometrica, Econometric Society, vol. 82(1), pages 137-196, January.
    13. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    14. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    15. Simon Burgess & Ellen Greaves & Anna Vignoles & Deborah Wilson, 2015. "What Parents Want: School Preferences and School Choice," Economic Journal, Royal Economic Society, vol. 125(587), pages 1262-1289, September.
    16. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    17. Vaart,A. W. van der, 2000. "Asymptotic Statistics," Cambridge Books, Cambridge University Press, number 9780521784504.
    18. C. Kirabo Jackson, 2010. "Do Students Benefit from Attending Better Schools? Evidence from Rule-based Student Assignments in Trinidad and Tobago," Economic Journal, Royal Economic Society, vol. 120(549), pages 1399-1429, December.
    19. Fuhito Kojima & Parag A. Pathak, 2009. "Incentives and Stability in Large Two-Sided Matching Markets," American Economic Review, American Economic Association, vol. 99(3), pages 608-627, June.
    20. Han Hong & Denis Nekipelov, 2010. "Semiparametric efficiency in nonlinear LATE models," Quantitative Economics, Econometric Society, vol. 1(2), pages 279-304, November.
    21. repec:mpr:mprres:6698 is not listed on IDEAS
    22. Yeon-Koo Che & Fuhito Kojima, 2010. "Asymptotic Equivalence of Probabilistic Serial and Random Priority Mechanisms," Econometrica, Econometric Society, vol. 78(5), pages 1625-1672, September.
    23. Atila Abdulkadiroğlu & Joshua D. Angrist & Peter D. Hull & Parag A. Pathak, 2016. "Charters without Lotteries: Testing Takeovers in New Orleans and Boston," American Economic Review, American Economic Association, vol. 106(7), pages 1878-1920, July.
    24. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    25. Bing-ru Teh & Moria McCullough & Brian P. Gill, "undated". "Student Achievement in New York City Middle Schools Affiliated with Achievement First and Uncommon Schools," Mathematica Policy Research Reports 9b7646f5c8ae4067840b186c5, Mathematica Policy Research.
    26. Atila Abdulkadiroğlu & Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak, 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 699-748.
    27. Atila Abdulkadiroğlu & Weiwei Hu & Parag A. Pathak, 2013. "Small High Schools and Student Achievement: Lottery-Based Evidence from New York City," NBER Working Papers 19576, National Bureau of Economic Research, Inc.
    28. Philip M. Gleason & Christina Clark Tuttle & Brian Gill & Ira Nichols-Barrer & Bing-ru Teh, 2014. "Do KIPP Schools Boost Student Achievement?," Education Finance and Policy, MIT Press, vol. 9(1), pages 36-58, January.
    29. Atila Abdulkadiroglu & Tayfun Sonmez, 1998. "Random Serial Dictatorship and the Core from Random Endowments in House Allocation Problems," Econometrica, Econometric Society, vol. 66(3), pages 689-702, May.
    30. Ergin, Haluk & Sonmez, Tayfun, 2006. "Games of school choice under the Boston mechanism," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 215-237, January.
    31. Howard S. Bloom & Rebecca Unterman, 2014. "Can Small High Schools of Choice Improve Educational Prospects for Disadvantaged Students?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 290-319, March.
    32. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    33. Newey, Whitney K, 1990. "Efficient Instrumental Variables Estimation of Nonlinear Models," Econometrica, Econometric Society, vol. 58(4), pages 809-837, July.
    34. Lars-Gunnar Svensson, 1999. "Strategy-proof allocation of indivisible goods," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 16(4), pages 557-567.
    35. Philip M. Gleason & Christina Clark Tuttle & Brian Gill & Ira Nichols-Barrer & Bing-ru Teh, 2014. "Do KIPP Schools Boost Student Achievement?," Mathematica Policy Research Reports 025c1b3af1c040ecbedfbce45, Mathematica Policy Research.
    36. Umut M. Dur & Scott Duke Kominers & Parag A. Pathak & Tayfun Sönmez, 2013. "The Demise of Walk Zones in Boston: Priorities vs. Precedence in School Choice," NBER Working Papers 18981, National Bureau of Economic Research, Inc.
    37. Eric Budish, 2011. "The Combinatorial Assignment Problem: Approximate Competitive Equilibrium from Equal Incomes," Journal of Political Economy, University of Chicago Press, vol. 119(6), pages 1061-1103.
    38. Umut Mert Dur & Parag A. Pathak & Tayfun Sönmez, 2016. "Explicit vs. Statistical Preferential Treatment in Affirmative Action: Theory and Evidence from Chicago’s Exam Schools," Boston College Working Papers in Economics 906, Boston College Department of Economics.
    39. Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
    40. repec:mpr:mprres:8002 is not listed on IDEAS
    41. Atila Abdulkadiroglu & Tayfun Sönmez, 2003. "School Choice: A Mechanism Design Approach," American Economic Review, American Economic Association, vol. 93(3), pages 729-747, June.
    42. repec:oup:restud:v:84:y::i:1:p:444-463. is not listed on IDEAS
    43. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2014. "Finite Population Causal Standard Errors," NBER Working Papers 20325, National Bureau of Economic Research, Inc.
    44. Joshua Angrist & Jinyong Hahn, 2004. "When to Control for Covariates? Panel Asymptotics for Estimates of Treatment Effects," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 58-72, February.
    45. Cristian Pop-Eleches & Miguel Urquiola, 2013. "Going to a Better School: Effects and Behavioral Responses," American Economic Review, American Economic Association, vol. 103(4), pages 1289-1324, June.
    46. Shu Yang & Guido W. Imbens & Zhanglin Cui & Douglas E. Faries & Zbigniew Kadziola, 2016. "Propensity score matching and subclassification in observational studies with multi‐level treatments," Biometrics, The International Biometric Society, vol. 72(4), pages 1055-1065, December.
    47. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    48. Eduardo M. Azevedo & Jacob D. Leshno, 2016. "A Supply and Demand Framework for Two-Sided Matching Markets," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1235-1268.
    49. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    50. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    3. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    4. Gabrielle Fack & Julien Grenet & Yinghua He, 2019. "Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions," American Economic Review, American Economic Association, vol. 109(4), pages 1486-1529, April.
    5. Angrist, Joshua D. & Pathak, Parag A. & Zarate, Roman A., 2023. "Choice and consequence: Assessing mismatch at Chicago exam schools," Journal of Public Economics, Elsevier, vol. 223(C).
    6. Yusuke Narita, 2021. "A Theory of Quasi-Experimental Evaluation of School Quality," Management Science, INFORMS, vol. 67(8), pages 4982-5010, August.
    7. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    8. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    9. Marco Ovidi, 2021. "Parents know better: primary school choice and student achievement in London," Working Papers 919, Queen Mary University of London, School of Economics and Finance.
    10. Atila Abdulkadiroğlu & Nikhil Agarwal & Parag A. Pathak, 2015. "The Welfare Effects of Coordinated Assignment: Evidence from the NYC HS Match," NBER Working Papers 21046, National Bureau of Economic Research, Inc.
    11. Oosterbeek, Hessel & Ruijs, Nienke & de Wolf, Inge, 2023. "Heterogeneous effects of comprehensive vs. single-track academic schools: Evidence from admission lotteries," Economics of Education Review, Elsevier, vol. 93(C).
    12. Diether W Beuermann & C Kirabo Jackson & Laia Navarro-Sola & Francisco Pardo, 2023. "What is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 65-101.
    13. Kirkebøen, Lars & Leuven, Edwin & Mogstad, Magne, 2014. "Field of Study, Earnings, and Self-Selection," Memorandum 29/2014, Oslo University, Department of Economics.
    14. Ha, Wei & Kang, Le & Song, Yang, 2020. "College matching mechanisms and matching stability: Evidence from a natural experiment in China," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 206-226.
    15. Thierry Magnac, 2018. "Quels étudiants pour quelles universités ? Analyses empiriques de mécanismes d’allocation centralisée," Revue économique, Presses de Sciences-Po, vol. 69(5), pages 683-708.
    16. Atila Abdulkadiroğlu & Weiwei Hu & Parag A. Pathak, 2013. "Small High Schools and Student Achievement: Lottery-Based Evidence from New York City," NBER Working Papers 19576, National Bureau of Economic Research, Inc.
    17. Joshua D. Angrist, 2022. "Empirical Strategies in Economics: Illuminating the Path From Cause to Effect," Econometrica, Econometric Society, vol. 90(6), pages 2509-2539, November.
    18. Ashlagi, Itai & Nikzad, Afshin & Romm, Assaf, 2019. "Assigning more students to their top choices: A comparison of tie-breaking rules," Games and Economic Behavior, Elsevier, vol. 115(C), pages 167-187.
    19. Akbarpour, Mohammad & Kapor, Adam & Neilson, Christopher & van Dijk, Winnie & Zimmerman, Seth, 2022. "Centralized School choice with unequal outside options," Journal of Public Economics, Elsevier, vol. 210(C).
    20. Hatfield, John William & Kojima, Fuhito & Narita, Yusuke, 2016. "Improving schools through school choice: A market design approach," Journal of Economic Theory, Elsevier, vol. 166(C), pages 186-211.

    More about this item

    JEL classification:

    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • I20 - Health, Education, and Welfare - - Education - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:21705. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.