IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v67y2021i8p4982-5010.html
   My bibliography  Save this article

A Theory of Quasi-Experimental Evaluation of School Quality

Author

Listed:
  • Yusuke Narita

    (Department of Economics and Cowles Foundation, Yale University, New Haven, Connecticut 06520)

Abstract

Many centralized school admissions systems use lotteries to ration limited seats at oversubscribed schools. The resulting random assignment is used by empirical researchers to identify the effects of schools on outcomes like test scores. I first find that the two most popular empirical research designs may not successfully extract a random assignment of applicants to schools. When are the research designs able to overcome this problem? I show the following main results for a class of data-generating mechanisms containing those used in practice: The first-choice research design extracts a random assignment under a mechanism if the mechanism is strategy-proof for schools. In contrast, the other qualification instrument research design does not necessarily extract a random assignment under any mechanism. The former research design is therefore more compelling than the latter. Many applications of the two research designs need some implicit assumption, such as large-sample approximately random assignment, to justify their empirical strategy.

Suggested Citation

  • Yusuke Narita, 2021. "A Theory of Quasi-Experimental Evaluation of School Quality," Management Science, INFORMS, vol. 67(8), pages 4982-5010, August.
    • Handle: RePEc:inm:ormnsc:v:67:y:2021:i:8:p:4982-5010
    DOI: 10.1287/mnsc.2020.3742
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.2020.3742
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.2020.3742?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2013. "Are Some Degrees Worth More than Others? Evidence from college admission cutoffs in Chile," NBER Working Papers 19241, National Bureau of Economic Research, Inc.
    2. Jackson, C. Kirabo, 2012. "Single-sex schools, student achievement, and course selection: Evidence from rule-based student assignments in Trinidad and Tobago," Journal of Public Economics, Elsevier, vol. 96(1), pages 173-187.
    3. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    4. David J. Deming, 2011. "Better Schools, Less Crime?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 2063-2115.
    5. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    6. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    7. Howard S. Bloom & Rebecca Unterman, 2014. "Can Small High Schools of Choice Improve Educational Prospects for Disadvantaged Students?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 290-319, March.
    8. Fort, Margherita & Zanella, Giulio, 2019. "Cognitive and non-cognitive costs of daycare 0–2 for children in advantaged families," CEPR Discussion Papers 11120, C.E.P.R. Discussion Papers.
    9. Yusuke Narita & Kohei Yata, 2021. "Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules," Working Papers 2021-022, Human Capital and Economic Opportunity Working Group.
    10. Caterina Calsamiglia & Chao Fu & Maia Güell, 2014. "Structural Estimation of a Model of School Choices: the Boston Mechanism vs. Its Alternatives," Working Papers 811, Barcelona School of Economics.
    11. Shapley, Lloyd & Scarf, Herbert, 1974. "On cores and indivisibility," Journal of Mathematical Economics, Elsevier, vol. 1(1), pages 23-37, March.
    12. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    13. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    14. , A. & ,, 2011. "Lotteries in student assignment: An equivalence result," Theoretical Economics, Econometric Society, vol. 6(1), January.
    15. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2012. "The Effect of School Choice on Intrinsic Motivation and Academic Outcomes," Working Papers 2012-3, Princeton University. Economics Department..
    16. David J. Deming & Justine S. Hastings & Thomas J. Kane & Douglas O. Staiger, 2014. "School Choice, School Quality, and Postsecondary Attainment," American Economic Review, American Economic Association, vol. 104(3), pages 991-1013, March.
    17. He, Yinghua, 2012. "Gaming the Boston School Choice Mechanism in Beijing," TSE Working Papers 12-345, Toulouse School of Economics (TSE).
    18. C. Kirabo Jackson, 2010. "Do Students Benefit from Attending Better Schools? Evidence from Rule-based Student Assignments in Trinidad and Tobago," Economic Journal, Royal Economic Society, vol. 120(549), pages 1399-1429, December.
    19. Fuhito Kojima & Parag A. Pathak, 2009. "Incentives and Stability in Large Two-Sided Matching Markets," American Economic Review, American Economic Association, vol. 99(3), pages 608-627, June.
    20. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    21. Yusuke Narita, 2021. "Incorporating ethics and welfare into randomized experiments," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(1), pages 2008740118-, January.
    22. Gary Chamberlain, 1980. "Analysis of Covariance with Qualitative Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 225-238.
    23. Hatfield, John William & Kojima, Fuhito & Narita, Yusuke, 2016. "Improving schools through school choice: A market design approach," Journal of Economic Theory, Elsevier, vol. 166(C), pages 186-211.
    24. Cristian Pop-Eleches & Miguel Urquiola, 2013. "Going to a Better School: Effects and Behavioral Responses," American Economic Review, American Economic Association, vol. 103(4), pages 1289-1324, June.
    25. Adam J. Kapor & Christopher A. Neilson & Seth D. Zimmerman, 2020. "Heterogeneous Beliefs and School Choice Mechanisms," American Economic Review, American Economic Association, vol. 110(5), pages 1274-1315, May.
    26. Chung-Piaw Teo & Jay Sethuraman & Wee-Peng Tan, 2001. "Gale-Shapley Stable Marriage Problem Revisited: Strategic Issues and Applications," Management Science, INFORMS, vol. 47(9), pages 1252-1267, September.
    27. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    28. Azevedo, Eduardo M., 2014. "Imperfect competition in two-sided matching markets," Games and Economic Behavior, Elsevier, vol. 83(C), pages 207-223.
    29. Eduardo M. Azevedo & Jacob D. Leshno, 2016. "A Supply and Demand Framework for Two-Sided Matching Markets," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1235-1268.
    30. Roth, Alvin E & Sotomayor, Marilda, 1989. "The College Admissions Problem Revisited," Econometrica, Econometric Society, vol. 57(3), pages 559-570, May.
    31. Elliott Peranson & Alvin E. Roth, 1999. "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design," American Economic Review, American Economic Association, vol. 89(4), pages 748-780, September.
    32. Yan Chen & Onur Kesten, 2017. "Chinese College Admissions and School Choice Reforms: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 125(1), pages 99-139.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    3. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    4. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    5. Gabrielle Fack & Julien Grenet & Yinghua He, 2019. "Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions," American Economic Review, American Economic Association, vol. 109(4), pages 1486-1529, April.
    6. Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Impact Evaluation in Matching Markets with General Tie-Breaking," NBER Working Papers 24172, National Bureau of Economic Research, Inc.
    7. Diether W Beuermann & C Kirabo Jackson & Laia Navarro-Sola & Francisco Pardo, 2023. "What is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 65-101.
    8. Abdulkadiroglu, Atila & Andersson, Tommy, 2022. "School Choice," Working Papers 2022:4, Lund University, Department of Economics.
    9. Cao, Yuan, 2020. "Centralized assignment mechanisms and assortative matching: Evidence from Chinese universities," Journal of Economic Behavior & Organization, Elsevier, vol. 177(C), pages 255-276.
    10. Li, Mengling, 2020. "Ties matter: Improving efficiency in course allocation by allowing ties," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 354-384.
    11. TANAKA Mari & NARITA Yusuke & MORIGUCHI Chiaki, 2020. "Meritocracy and Its Discontent: Long-run Effects of Repeated School Admission Reforms," Discussion papers 20002, Research Institute of Economy, Trade and Industry (RIETI).
    12. Alvin Roth, 2008. "Deferred acceptance algorithms: history, theory, practice, and open questions," International Journal of Game Theory, Springer;Game Theory Society, vol. 36(3), pages 537-569, March.
    13. Ha, Wei & Kang, Le & Song, Yang, 2020. "College matching mechanisms and matching stability: Evidence from a natural experiment in China," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 206-226.
    14. John William Hatfield & Fuhito Kojima & Yusuke Narita, 2011. "Promoting School Competition Through School Choice: A Market Design Approach," Working Papers 2011-018, Human Capital and Economic Opportunity Working Group.
    15. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    16. Artemov, Georgy, 2021. "Assignment mechanisms: Common preferences and information acquisition," Journal of Economic Theory, Elsevier, vol. 198(C).
    17. Thierry Magnac, 2018. "Quels étudiants pour quelles universités ? Analyses empiriques de mécanismes d’allocation centralisée," Revue économique, Presses de Sciences-Po, vol. 69(5), pages 683-708.
    18. Aaron L. Bodoh-Creed, 2020. "Optimizing for Distributional Goals in School Choice Problems," Management Science, INFORMS, vol. 66(8), pages 3657-3676, August.
    19. Marco Ovidi, 2022. "Parents Know Better: Sorting on Match Effects in Primary School," DISCE - Working Papers del Dipartimento di Economia e Finanza def121, Università Cattolica del Sacro Cuore, Dipartimenti e Istituti di Scienze Economiche (DISCE).
    20. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:67:y:2021:i:8:p:4982-5010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.