IDEAS home Printed from https://ideas.repec.org/a/bpj/bejeap/v15y2015i3p1417-1453n6.html
   My bibliography  Save this article

Noise or News? Learning about the Content of Test-Based School Achievement Measures

Author

Listed:
  • Javdani Mohsen

    (Department of Economics, University of British Columbia, Okanagan Campus, Kelowna, BC, Canada V1V 1V7)

Abstract

Test-based measures of school performance are increasingly used to inform both education policy and families’ school choice decisions. There are, however, concerns about the reliability of these measures. This paper assesses the extent to which cross-sectional differences in schools’ average achievement on standardized tests reflect transitory factors, using data from a 1999–2006 panel of public and private schools in British Columbia, Canada. Sampling variation and one-time mean reverting shocks are shown to be significant sources of cross-sectional variation in schools’ mean test scores. The results therefore suggest that public dissemination of information about schools’ average achievement on standardized tests could potentially mislead or confuse parents. These results should also encourage caution in the design of policies that attach monetary or nonmonetary rewards or sanctions to schools on the basis of test-based achievement and suggest a need for more sophisticated measures of school performance.

Suggested Citation

  • Javdani Mohsen, 2015. "Noise or News? Learning about the Content of Test-Based School Achievement Measures," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(3), pages 1417-1453, July.
    • Handle: RePEc:bpj:bejeap:v:15:y:2015:i:3:p:1417-1453:n:6
    DOI: 10.1515/bejeap-2014-0052
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/bejeap-2014-0052
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/bejeap-2014-0052?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Allen, Rebecca & Burgess, Simon, 2013. "Evaluating the provision of school performance information for school choice," Economics of Education Review, Elsevier, vol. 34(C), pages 175-190.
    2. Robert Bifulco & Helen F. Ladd, 2006. "The Impacts of Charter Schools on Student Achievement: Evidence from North Carolina," Education Finance and Policy, MIT Press, vol. 1(1), pages 50-90, January.
    3. Tülin Erdem & Michael P. Keane, 1996. "Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets," Marketing Science, INFORMS, vol. 15(1), pages 1-20.
    4. Borland, Melvin V. & Howsen, Roy M, 1992. "Student academic achievement and the degree of market concentration in education," Economics of Education Review, Elsevier, vol. 11(1), pages 31-39, March.
    5. Julie Berry Cullen & Brian A Jacob & Steven Levitt, 2006. "The Effect of School Choice on Participants: Evidence from Randomized Lotteries," Econometrica, Econometric Society, vol. 74(5), pages 1191-1230, September.
    6. Kenneth Y. Chay & Patrick J. McEwan & Miguel Urquiola, 2005. "The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools," American Economic Review, American Economic Association, vol. 95(4), pages 1237-1258, September.
    7. Cullen, Julie Berry & Jacob, Brian A. & Levitt, Steven D., 2005. "The impact of school choice on student outcomes: an analysis of the Chicago Public Schools," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 729-760, June.
    8. Chernew, Michael & Gowrisankaran, Gautam & Scanlon, Dennis P., 2008. "Learning and the value of information: Evidence from health plan report cards," Journal of Econometrics, Elsevier, vol. 144(1), pages 156-174, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Benjamin Cerf Harris, 2014. "Within and Across County Variation in SNAP Misreporting: Evidence from Linked ACS and Administrative Records," CARRA Working Papers 2014-05, Center for Economic Studies, U.S. Census Bureau.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    2. Nirav Mehta, 2017. "Competition In Public School Districts: Charter School Entry, Student Sorting, And School Input Determination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 58(4), pages 1089-1116, November.
    3. Akyol, Pelin & Krishna, Kala, 2017. "Preferences, selection, and value added: A structural approach," European Economic Review, Elsevier, vol. 91(C), pages 89-117.
    4. Miguel Urquiola, 2015. "Progress and challenges in achieving an evidence-based education policy in Latin America and the Caribbean," Latin American Economic Review, Springer;Centro de Investigaciòn y Docencia Económica (CIDE), vol. 24(1), pages 1-30, December.
    5. Gaurav Khanna & Carlos Medina & Anant Nyshadham & Jorge Tamayo & Nicolas Torres, 2023. "Formal Employment and Organised Crime: Regression Discontinuity Evidence from Colombia," The Economic Journal, Royal Economic Society, vol. 133(654), pages 2427-2448.
    6. Scott A. Imberman, 2011. "Achievement and Behavior in Charter Schools: Drawing a More Complete Picture," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 416-435, May.
    7. Böhlmark, Anders & Lindahl, Mikael, 2007. "The Impact of School Choice on Pupil Achievement, Segregation and Costs: Swedish Evidence," IZA Discussion Papers 2786, Institute of Labor Economics (IZA).
    8. David Card & Martin D. Dooley & A. Abigail Payne, 2010. "School Competition and Efficiency with Publicly Funded Catholic Schools," American Economic Journal: Applied Economics, American Economic Association, vol. 2(4), pages 150-176, October.
    9. Friesen, Jane & Harris, Benjamin Cerf & Woodcock, Simon, 2013. "Open Enrolment and Student Achievement," CLSSRN working papers clsrn_admin-2013-46, Vancouver School of Economics, revised 22 Mar 2014.
    10. Dennis Epple & Richard E. Romano & Miguel Urquiola, 2017. "School Vouchers: A Survey of the Economics Literature," Journal of Economic Literature, American Economic Association, vol. 55(2), pages 441-492, June.
    11. Will Dobbie & Roland G. Fryer Jr., 2013. "Getting beneath the Veil of Effective Schools: Evidence from New York City," American Economic Journal: Applied Economics, American Economic Association, vol. 5(4), pages 28-60, October.
    12. Masi, Barbara, 2018. "A ticket to ride: The unintended consequences of school transport subsidies," Economics of Education Review, Elsevier, vol. 63(C), pages 100-115.
    13. Jackson, C. Kirabo, 2013. "Can higher-achieving peers explain the benefits to attending selective schools? Evidence from Trinidad and Tobago," Journal of Public Economics, Elsevier, vol. 108(C), pages 63-77.
    14. Jesse M. Rothstein, 2006. "Good Principals or Good Peers? Parental Valuation of School Characteristics, Tiebout Equilibrium, and the Incentive Effects of Competition among Jurisdictions," American Economic Review, American Economic Association, vol. 96(4), pages 1333-1350, September.
    15. repec:mpr:mprres:5572 is not listed on IDEAS
    16. Kim, Youngran, 2018. "The effects of school choice on achievement gaps between private and public high schools: Evidence from the Seoul high school choice program," International Journal of Educational Development, Elsevier, vol. 60(C), pages 25-32.
    17. Laura Grube & Devin Anderson, 2018. "School Choice and Charter Schools in Review: What Have We Learned?," Journal of Private Enterprise, The Association of Private Enterprise Education, vol. 33(Winter 20), pages 21-44.
    18. Burgess, Simon, 2016. "Human Capital and Education: The State of the Art in the Economics of Education," IZA Discussion Papers 9885, Institute of Labor Economics (IZA).
    19. Lai, Fang & Sadoulet, Elisabeth & de Janvry, Alain, 2009. "The adverse effects of parents' school selection errors on academic achievement: Evidence from the Beijing open enrollment program," Economics of Education Review, Elsevier, vol. 28(4), pages 485-496, August.
    20. Welsch, David M. & Zimmer, David M., 2012. "Do student migrations affect school performance? Evidence from Wisconsin's inter-district public school program," Economics of Education Review, Elsevier, vol. 31(1), pages 195-207.
    21. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:bejeap:v:15:y:2015:i:3:p:1417-1453:n:6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.