IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/18038.html
   My bibliography  Save this paper

Knowledge, Tests, and Fadeout in Educational Interventions

Author

Listed:
  • Elizabeth U. Cascio
  • Douglas O. Staiger

Abstract

Educational interventions are often evaluated and compared on the basis of their impacts on test scores. Decades of research have produced two empirical regularities: interventions in later grades tend to have smaller effects than the same interventions in earlier grades, and the test score impacts of early educational interventions almost universally "fade out" over time. This paper explores whether these empirical regularities are an artifact of the common practice of rescaling test scores in terms of a student's position in a widening distribution of knowledge. If a standard deviation in test scores in later grades translates into a larger difference in knowledge, an intervention's effect on normalized test scores may fall even as its effect on knowledge does not. We evaluate this hypothesis by fitting a model of education production to correlations in test scores across grades and with college-going using both administrative and survey data. Our results imply that the variance in knowledge does indeed rise as children progress through school, but not enough for test score normalization to fully explain these empirical regularities.

Suggested Citation

  • Elizabeth U. Cascio & Douglas O. Staiger, 2012. "Knowledge, Tests, and Fadeout in Educational Interventions," NBER Working Papers 18038, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:18038
    Note: CH ED LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w18038.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jesse Rothstein, 2010. "Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 175-214.
    2. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    3. Raj Chetty & John N. Friedman & Nathaniel Hilger & Emmanuel Saez & Diane Whitmore Schanzenbach & Danny Yagan, 2011. "How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1593-1660.
    4. Timothy N. Bond & Kevin Lang, 2013. "The Evolution of the Black-White Test Score Gap in Grades K–3: The Fragility of Results," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1468-1479, December.
    5. Kevin Lang, 2010. "Measurement Matters: Perspectives on Education Policy from an Economist and School Board Member," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 167-182, Summer.
    6. Flavio Cunha & James J. Heckman, 2008. "Formulating, Identifying and Estimating the Technology of Cognitive and Noncognitive Skill Formation," Journal of Human Resources, University of Wisconsin Press, vol. 43(4).
    7. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    8. Neal, Derek A & Johnson, William R, 1996. "The Role of Premarket Factors in Black-White Wage Differences," Journal of Political Economy, University of Chicago Press, vol. 104(5), pages 869-895, October.
    9. Krueger, Alan B & Whitmore, Diane M, 2001. "The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR," Economic Journal, Royal Economic Society, vol. 111(468), pages 1-28, January.
    10. Will Dobbie & Roland G. Fryer, 2011. "Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children's Zone," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 158-187, July.
    11. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2011. "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood," NBER Working Papers 17699, National Bureau of Economic Research, Inc.
    12. Elizabeth U. Cascio & Ethan G. Lewis, 2006. "Schooling and the Armed Forces Qualifying Test: Evidence from School-Entry Laws," Journal of Human Resources, University of Wisconsin Press, vol. 41(2).
    13. Daniel F. McCaffrey & J. R. Lockwood & Daniel Koretz & Thomas A. Louis & Laura Hamilton, 2004. "Models for Value-Added Modeling of Teacher Effects," Journal of Educational and Behavioral Statistics, , vol. 29(1), pages 67-101, March.
    14. John Cawley & James Heckman & Edward Vytlacil, 1999. "On Policies To Reward The Value Added By Educators," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 720-727, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sean Corcoran & Dan Goldhaber, 2013. "Value Added and Its Uses: Where You Stand Depends on Where You Sit," Education Finance and Policy, MIT Press, vol. 8(3), pages 418-434, July.
    2. repec:hrv:faseco:30749606 is not listed on IDEAS
    3. Shaun M. Dougherty, 2015. "Bridging the Discontinuity in Adolescent Literacy? Mixed Evidence from a Middle Grades Intervention," Education Finance and Policy, MIT Press, vol. 10(2), pages 157-192, March.
    4. Philip J. Cook & Kenneth Dodge & George Farkas & Roland G. Fryer, Jr & Jonathan Guryan & Jens Ludwig & Susan Mayer & Harold Pollack & Laurence Steinberg, 2014. "The (Surprising) Efficacy of Academic and Behavioral Intervention with Disadvantaged Youth: Results from a Randomized Experiment in Chicago," NBER Working Papers 19862, National Bureau of Economic Research, Inc.
    5. Duflo, Esther & Dupas, Pascaline & Kremer, Michael, 2015. "School governance, teacher incentives, and pupil–teacher ratios: Experimental evidence from Kenyan primary schools," Journal of Public Economics, Elsevier, vol. 123(C), pages 92-110.
    6. Wan, Sirui & Bond, Timothy N. & Lang, Kevin & Clements, Douglas H. & Sarama, Julie & Bailey, Drew H., 2021. "Is intervention fadeout a scaling artefact?," Economics of Education Review, Elsevier, vol. 82(C).
    7. Carsten Schroeder & Shlomo Yitzhaki, 2020. "Exploring the robustness of country rankings by educational attainment," Journal of Economics, Springer, vol. 129(3), pages 271-296, April.
    8. Schwerdt, Guido & West, Martin R. & Winters, Marcus A., 2017. "The effects of test-based retention on student outcomes over time: Regression discontinuity evidence from Florida," Journal of Public Economics, Elsevier, vol. 152(C), pages 154-169.
    9. Elizabeth U. Cascio & Diane Whitmore Schanzenbach, 2016. "First in the Class? Age and the Education Production Function," Education Finance and Policy, MIT Press, vol. 11(3), pages 225-250, Summer.
    10. Jeffrey Penney, 2017. "Racial Interaction Effects and Student Achievement," Education Finance and Policy, MIT Press, vol. 12(4), pages 447-467, Fall.
    11. Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    12. Douglas Almond & Bhashkar Mazumder & Reyn Ewijk, 2015. "In Utero Ramadan Exposure and Children's Academic Performance," Economic Journal, Royal Economic Society, vol. 125(589), pages 1501-1533, December.
    13. Julie Berry Cullen & Steven D. Levitt & Erin Robertson & Sally Sadoff, 2013. "What Can Be Done to Improve Struggling High Schools?," Journal of Economic Perspectives, American Economic Association, vol. 27(2), pages 133-152, Spring.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    2. Koedel, Cory & Mihaly, Kata & Rockoff, Jonah E., 2015. "Value-added modeling: A review," Economics of Education Review, Elsevier, vol. 47(C), pages 180-195.
    3. Dan Goldhaber & Michael Hansen, 2013. "Is it Just a Bad Class? Assessing the Long-term Stability of Estimated Teacher Performance," Economica, London School of Economics and Political Science, vol. 80(319), pages 589-612, July.
    4. Seth Gershenson & Diane Whitmore Schanzenbach, 2016. "Linking Teacher Quality, Student Attendance, and Student Achievement," Education Finance and Policy, MIT Press, vol. 11(2), pages 125-149, Spring.
    5. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja & Tristan Zajonc, 2011. "Do Value-Added Estimates Add Value? Accounting for Learning Dynamics," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 29-54, July.
    6. Koedel Cory & Leatherman Rebecca & Parsons Eric, 2012. "Test Measurement Error and Inference from Value-Added Models," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 12(1), pages 1-37, November.
    7. Bhalotra, Sonia R. & Karlsson, Martin & Nilsson, Therese & Schwarz, Nina, 2016. "Infant Health, Cognitive Performance and Earnings: Evidence from Inception of the Welfare State in Sweden," IZA Discussion Papers 10339, Institute of Labor Economics (IZA).
    8. Stacy, Brian & Guarino, Cassandra & Wooldridge, Jeffrey, 2018. "Does the precision and stability of value-added estimates of teacher performance depend on the types of students they serve?," Economics of Education Review, Elsevier, vol. 64(C), pages 50-74.
    9. Vilsa E. Curto & Roland G. Fryer Jr., 2014. "The Potential of Urban Boarding Schools for the Poor: Evidence from SEED," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 65-93.
    10. C. Kirabo Jackson, 2012. "Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina," NBER Working Papers 18624, National Bureau of Economic Research, Inc.
    11. Douglas Almond & Bhashkar Mazumder & Reyn Ewijk, 2015. "In Utero Ramadan Exposure and Children's Academic Performance," Economic Journal, Royal Economic Society, vol. 125(589), pages 1501-1533, December.
    12. Roland G. Fryer, Jr, 2016. "The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments," NBER Working Papers 22130, National Bureau of Economic Research, Inc.
    13. Hanushek, Eric A., 2011. "The economic value of higher teacher quality," Economics of Education Review, Elsevier, vol. 30(3), pages 466-479, June.
    14. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    15. Cassandra M. Guarino & Mark D. Reckase & Jeffrey M. Woolrdige, 2014. "Can Value-Added Measures of Teacher Performance Be Trusted?," Education Finance and Policy, MIT Press, vol. 10(1), pages 117-156, November.
    16. Marine de Talancé, 2015. "Better Teachers, Better Results? Evidence from Rural Pakistan," Working Papers DT/2015/21, DIAL (Développement, Institutions et Mondialisation).
    17. Dieterle, Steven G., 2015. "Class-size reduction policies and the quality of entering teachers," Labour Economics, Elsevier, vol. 36(C), pages 35-47.
    18. Nirav Mehta, 2014. "Targeting the Wrong Teachers: Estimating Teacher Quality for Use in Accountability Regimes," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20143, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
    19. Taylor, Eric, 2014. "Spending more of the school day in math class: Evidence from a regression discontinuity in middle school," Journal of Public Economics, Elsevier, vol. 117(C), pages 162-181.
    20. Murguia Baysse, Juan Manuel, 2013. "Essays on agricultural, financial economics and education," ISU General Staff Papers 201301010800004458, Iowa State University, Department of Economics.

    More about this item

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:18038. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.