IDEAS home Printed from https://ideas.repec.org/a/oup/qjecon/v132y2017i2p871-919..html
   My bibliography  Save this article

Leveraging Lotteries for School Value-Added: Testing and Estimation

Author

Listed:
  • Joshua D. Angrist
  • Peter D. Hull
  • Parag A. Pathak
  • Christopher R. Walters

Abstract

Conventional value-added models (VAMs) compare average test scores across schools after regression-adjusting for students’ demographic characteristics and previous scores. This article tests for VAM bias using a procedure that asks whether VAM estimates accurately predict the achievement consequences of random assignment to specific schools. Test results from admissions lotteries in Boston suggest conventional VAM estimates are biased, a finding that motivates the development of a hierarchical model describing the joint distribution of school value-added, bias, and lottery compliance. We use this model to assess the substantive importance of bias in conventional VAM estimates and to construct hybrid value-added estimates that optimally combine ordinary least squares and lottery-based estimates of VAM parameters. The hybrid estimation strategy provides a general recipe for combining nonexperimental and quasi-experimental estimates. While still biased, hybrid school value-added estimates have lower mean squared error than conventional VAM estimates. Simulations calibrated to the Boston data show that, bias notwithstanding, policy decisions based on conventional VAMs that control for lagged achievement are likely to generate substantial achievement gains. Hybrid estimates that incorporate lotteries yield further gains.

Suggested Citation

  • Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, Oxford University Press, vol. 132(2), pages 871-919.
  • Handle: RePEc:oup:qjecon:v:132:y:2017:i:2:p:871-919.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/qje/qjx001
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1593-1660.
    2. McFadden, Daniel, 1989. "A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration," Econometrica, Econometric Society, vol. 57(5), pages 995-1026, September.
    3. David J. Deming, 2014. "Using School Choice Lotteries to Test Measures of School Effectiveness," American Economic Review, American Economic Association, vol. 104(5), pages 406-411, May.
    4. Josh Kinsler, 2012. "Assessing Rothstein's critique of teacher value‐added models," Quantitative Economics, Econometric Society, vol. 3(2), pages 333-362, July.
    5. Cory Koedel & Julian R. Betts, 2011. "Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique," Education Finance and Policy, MIT Press, vol. 6(1), pages 18-42, January.
    6. Atila Abdulkadiroğlu & Joshua D. Angrist & Peter D. Hull & Parag A. Pathak, 2016. "Charters without Lotteries: Testing Takeovers in New Orleans and Boston," American Economic Review, American Economic Association, vol. 106(7), pages 1878-1920, July.
    7. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    8. Judge G.G. & Mittelhammer R.C., 2004. "A Semiparametric Basis for Combining Estimation Problems Under Quadratic Loss," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 479-487, January.
    9. Atila Abdulkadiroğlu & Joshua D. Angrist & Susan M. Dynarski & Thomas J. Kane & Parag A. Pathak, 2011. "Accountability and Flexibility in Public Schools: Evidence from Boston's Charters And Pilots," The Quarterly Journal of Economics, Oxford University Press, vol. 126(2), pages 699-748.
    Full references (including those not matched with items on IDEAS)

    More about this item

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:qjecon:v:132:y:2017:i:2:p:871-919.. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Oxford University Press). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.