IDEAS home Printed from https://ideas.repec.org/p/cwl/cwldpp/2056r.html
   My bibliography  Save this paper

(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality

Author

Abstract

Many centralized school admissions systems use lotteries to ration limited seats at oversubscribed schools. The resulting random assignment is used by empirical researchers to identify the effect of entering a school on outcomes like test scores. I first find that the two most popular empirical research designs may not successfully extract a random assignment of applicants to schools. When do the research designs overcome this problem" I show the following main results for a class of data-generating mechanisms containing those used in practice: One research design extracts a random assignment under a mechanism if and practically only if the mechanism is strategy-proof for schools. In contrast, the other research design does not necessarily extract a random assignment under any mechanism.

Suggested Citation

  • Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    • Handle: RePEc:cwl:cwldpp:2056r
    as

    Download full text from publisher

    File URL: https://cowles.yale.edu/sites/default/files/files/pub/d20/d2056-r.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    2. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    3. Fort, Margherita & Zanella, Giulio, 2019. "Cognitive and non-cognitive costs of daycare 0–2 for children in advantaged families," CEPR Discussion Papers 11120, C.E.P.R. Discussion Papers.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    5. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2012. "The Effect of School Choice on Intrinsic Motivation and Academic Outcomes," Working Papers 2012-3, Princeton University. Economics Department..
    6. Atila Abdulkadiroğlu & Joshua Angrist & Parag Pathak, 2014. "The Elite Illusion: Achievement Effects at Boston and New York Exam Schools," Econometrica, Econometric Society, vol. 82(1), pages 137-196, January.
    7. C. Kirabo Jackson, 2010. "Do Students Benefit from Attending Better Schools? Evidence from Rule-based Student Assignments in Trinidad and Tobago," Economic Journal, Royal Economic Society, vol. 120(549), pages 1399-1429, December.
    8. Fuhito Kojima & Parag A. Pathak, 2009. "Incentives and Stability in Large Two-Sided Matching Markets," American Economic Review, American Economic Association, vol. 99(3), pages 608-627, June.
    9. Lars Kirkebøen & Edwin Leuven & Magne Mogstad, 2014. "Field of Study, Earnings, and Self-Selection," NBER Working Papers 20816, National Bureau of Economic Research, Inc.
    10. Gary Chamberlain, 1980. "Analysis of Covariance with Qualitative Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 225-238.
    11. Diether Beuermann & C. Kirabo Jackson & Ricardo Sierra, 2015. "Privately Managed Public Secondary Schools and Academic Achievement in Trinidad and Tobago: Evidence from Rule-Based Student Assignments," IDB Publications (Working Papers) 91836, Inter-American Development Bank.
    12. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    13. Roth, Alvin E & Sotomayor, Marilda, 1989. "The College Admissions Problem Revisited," Econometrica, Econometric Society, vol. 57(3), pages 559-570, May.
    14. Peter Leopold S. Bergman, 2016. "The Effects of School Integration: Evidence from a Randomized Desegregation Program," CESifo Working Paper Series 6119, CESifo.
    15. Ergin, Haluk & Sonmez, Tayfun, 2006. "Games of school choice under the Boston mechanism," Journal of Public Economics, Elsevier, vol. 90(1-2), pages 215-237, January.
    16. Howard S. Bloom & Rebecca Unterman, 2014. "Can Small High Schools of Choice Improve Educational Prospects for Disadvantaged Students?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 290-319, March.
    17. Shapley, Lloyd & Scarf, Herbert, 1974. "On cores and indivisibility," Journal of Mathematical Economics, Elsevier, vol. 1(1), pages 23-37, March.
    18. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    19. Atila Abdulkadiroglu & Tayfun Sönmez, 2003. "School Choice: A Mechanism Design Approach," American Economic Review, American Economic Association, vol. 93(3), pages 729-747, June.
    20. repec:oup:restud:v:84:y::i:1:p:444-463. is not listed on IDEAS
    21. Beuermann, Diether & Jackson, C. Kirabo & Sierra, Ricardo, 2015. "Privately Managed Public Secondary Schools and Academic Achievement in Trinidad and Tobago: Evidence from Rule-Based Student Assignments," IDB Publications (Working Papers) 7308, Inter-American Development Bank.
    22. Hatfield, John William & Kojima, Fuhito & Narita, Yusuke, 2016. "Improving schools through school choice: A market design approach," Journal of Economic Theory, Elsevier, vol. 166(C), pages 186-211.
    23. Cristian Pop-Eleches & Miguel Urquiola, 2013. "Going to a Better School: Effects and Behavioral Responses," American Economic Review, American Economic Association, vol. 103(4), pages 1289-1324, June.
    24. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    25. Elliott Peranson & Alvin E. Roth, 1999. "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design," American Economic Review, American Economic Association, vol. 89(4), pages 748-780, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yusuke Narita, 2018. "Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials," Cowles Foundation Discussion Papers 2127r, Cowles Foundation for Research in Economics, Yale University, revised May 2019.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Yusuke Narita, 2021. "A Theory of Quasi-Experimental Evaluation of School Quality," Management Science, INFORMS, vol. 67(8), pages 4982-5010, August.
    3. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    4. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    5. Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Impact Evaluation in Matching Markets with General Tie-Breaking," NBER Working Papers 24172, National Bureau of Economic Research, Inc.
    6. John William Hatfield & Fuhito Kojima & Yusuke Narita, 2011. "Promoting School Competition Through School Choice: A Market Design Approach," Working Papers 2011-018, Human Capital and Economic Opportunity Working Group.
    7. Alvin Roth, 2008. "Deferred acceptance algorithms: history, theory, practice, and open questions," International Journal of Game Theory, Springer;Game Theory Society, vol. 36(3), pages 537-569, March.
    8. Eduardo M. Azevedo & Jacob D. Leshno, 2016. "A Supply and Demand Framework for Two-Sided Matching Markets," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1235-1268.
    9. Haeringer, Guillaume & Klijn, Flip, 2009. "Constrained school choice," Journal of Economic Theory, Elsevier, vol. 144(5), pages 1921-1947, September.
    10. Zhu, Min, 2014. "College admissions in China: A mechanism design perspective," China Economic Review, Elsevier, vol. 30(C), pages 618-631.
    11. Diether W Beuermann & C Kirabo Jackson & Laia Navarro-Sola & Francisco Pardo, 2023. "What is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 65-101.
    12. Alvin E. Roth, 2009. "What Have We Learned from Market Design?," Innovation Policy and the Economy, University of Chicago Press, vol. 9(1), pages 79-112.
    13. Committee, Nobel Prize, 2012. "Alvin E. Roth and Lloyd S. Shapley: Stable allocations and the practice of market design," Nobel Prize in Economics documents 2012-1, Nobel Prize Committee.
    14. Cao, Yuan, 2020. "Centralized assignment mechanisms and assortative matching: Evidence from Chinese universities," Journal of Economic Behavior & Organization, Elsevier, vol. 177(C), pages 255-276.
    15. Hatfield, John William & Kojima, Fuhito & Narita, Yusuke, 2016. "Improving schools through school choice: A market design approach," Journal of Economic Theory, Elsevier, vol. 166(C), pages 186-211.
    16. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    17. Min Zhu, 2013. "College Admissions in China : A Mechanism Design Perspective," Working Papers halshs-00860931, HAL.
    18. Oosterbeek, Hessel & Ruijs, Nienke & de Wolf, Inge, 2023. "Heterogeneous effects of comprehensive vs. single-track academic schools: Evidence from admission lotteries," Economics of Education Review, Elsevier, vol. 93(C).
    19. Min Zhu, 2013. "College Admissions in China : A Mechanism Design Perspective," Working Papers 1327, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    20. Kamada, Yuichiro & Kojima, Fuhito, 2017. "Stability concepts in matching under distributional constraints," Journal of Economic Theory, Elsevier, vol. 168(C), pages 107-142.

    More about this item

    Keywords

    Matching Market Design; Natural Experiment; Program Evaluation; Random Assignment; Quasi-Experimental Research Design; School Eectiveness;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:2056r. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Brittany Ladd (email available below). General contact details of provider: https://edirc.repec.org/data/cowleus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.