IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v68y2019icp89-103.html

Is the pen mightier than the keyboard? The effect of online testing on measured student achievement

Author

Listed:
  • Backes, Ben
  • Cowan, James

Abstract

Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.

Suggested Citation

  • Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    • Handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103
    DOI: 10.1016/j.econedurev.2018.12.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775718305119
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2018.12.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    2. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    3. Elizabeth U. Cascio & Douglas O. Staiger, 2012. "Knowledge, Tests, and Fadeout in Educational Interventions," NBER Working Papers 18038, National Bureau of Economic Research, Inc.
    4. Black, Sandra E. & Machin, Stephen, 2011. "Housing Valuations of School Performance," Handbook of the Economics of Education, in: Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), Handbook of the Economics of Education, edition 1, volume 3, chapter 10, pages 485-519, Elsevier.
    5. Backes, Ben & Cowan, James & Goldhaber, Dan & Koedel, Cory & Miller, Luke C. & Xu, Zeyu, 2018. "The common core conundrum: To what extent should we worry that changes to assessments will affect test-based measures of teacher performance?," Economics of Education Review, Elsevier, vol. 62(C), pages 48-65.
    6. Eric A. Hanushek & Steven G. Rivkin, 2010. "Generalizations about Using Value-Added Measures of Teacher Quality," American Economic Review, American Economic Association, vol. 100(2), pages 267-271, May.
    7. Markus Frolich & Blaise Melly, 2010. "Estimation of quantile treatment effects with Stata," Stata Journal, StataCorp LLC, vol. 10(3), pages 423-457, September.
    8. Kim, Jaehoon & Kim, Sangsin, 2015. "2012년 국회법 개정의 효과 연구 [A Study on the Effect of the 2012 National Assembly Act Amendment]," KDI Research Monographs, Korea Development Institute (KDI), volume 127, number v:2015-03(k):y:2015:p:1-1.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Freitas, Pedro & Nunes, Luis Catela & Reis, Ana B & Pereira dos Santos, João, 2025. "Can the Keyboard Beat the Good Old Pen? Computer-Based Testing and Students’ Performance," IZA Discussion Papers 18283, IZA Network @ LISER.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donna Feir & Kelly Foley & Maggie E. C. Jones, 2021. "The Distributional Impacts of Active Labor Market Programs for Indigenous Populations," AEA Papers and Proceedings, American Economic Association, vol. 111, pages 216-220, May.
    2. M. Fort, 2012. "Unconditional and Conditional Quantile Treatment Effect: Identification Strategies and Interpretations," Working Papers wp857, Dipartimento Scienze Economiche, Universita' di Bologna.
    3. Evangelista, Rui & Silva, João Andrade e & Ramalho, Esmeralda A., 2022. "How heterogeneous is the impact of energy efficiency on dwelling prices? Evidence from the application of the unconditional quantile hedonic model to the Portuguese residential market," Energy Economics, Elsevier, vol. 109(C).
    4. Laetitia Duval & François-Charles Wolff, 2013. "The consumption-enhancing effect of remittances: Evidence from Kosovo," wiiw Balkan Observatory Working Papers 107, The Vienna Institute for International Economic Studies, wiiw.
    5. Magbondé, Kadoukpè Gildas & Reilly, Barry & Kauma, Bridget, 2024. "The distributional effects of rural electrification on household welfare: Evidence from Senegal," Utilities Policy, Elsevier, vol. 91(C).
    6. Julie Berry Cullen & Cory Koedel & Eric Parsons, 2021. "The Compositional Effect of Rigorous Teacher Evaluation on Workforce Quality," Education Finance and Policy, MIT Press, vol. 16(1), pages 7-41, Winter.
    7. Francesca Caselli & Mr. Philippe Wingender, 2018. "Bunching at 3 Percent: The Maastricht Fiscal Criterion and Government Deficits," IMF Working Papers 2018/182, International Monetary Fund.
    8. Andrew Henley, 2023. "Is Rising Self-Employment Associated with Material Deprivation in the UK?," Work, Employment & Society, British Sociological Association, vol. 37(5), pages 1395-1418, October.
    9. Alex Coad & Julian Frankish & Paul Nightingale & Richard Roberts, 2014. "Business experience and start-up size: Buying more lottery tickets next time around?," Small Business Economics, Springer, vol. 43(3), pages 529-547, October.
    10. Zongwu Cai & Ying Fang & Ming Lin & Shengfang Tang, 2021. "A Nonparametric Test for Testing Heterogeneity in Conditional Quantile Treatment Effects," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202117, University of Kansas, Department of Economics, revised Aug 2021.
    11. Daly, Moira, 2015. "The long term returns of attempting self-employment with regular employment as a fall back option," Labour Economics, Elsevier, vol. 35(C), pages 26-52.
    12. Swaibu, Mbowa & Tonny, Odokonyero & Ezra, Munyambonera, 2014. "The potential of coffee to uplift people out of poverty in Northern Uganda," Research Reports 206171, Economic Policy Research Centre (EPRC).
    13. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    14. Damian Clarke & Manuel Llorca Jaña & Daniel Pailañir, 2023. "The use of quantile methods in economic history," Historical Methods: A Journal of Quantitative and Interdisciplinary History, Taylor & Francis Journals, vol. 56(2), pages 115-132, April.
    15. Bazzi, Samuel & Sumarto, Sudarno & Suryahadi, Asep, 2013. "It's All in the Timing:Household Expenditure and Labor Supply Responses to Unconditional Cash Transfers," MPRA Paper 57892, University Library of Munich, Germany, revised 31 Nov 2013.
    16. Schiltz, Fritz & Sestito, Paolo & Agasisti, Tommaso & De Witte, Kristof, 2018. "The added value of more accurate predictions for school rankings," Economics of Education Review, Elsevier, vol. 67(C), pages 207-215.
    17. Borgen, Nicolai T. & Haupt, Andreas & Wiborg, Øyvind N., 2021. "Flexible and fast estimation of quantile treatment effects: The rqr and rqrplot commands," SocArXiv 4vquh, Center for Open Science.
    18. repec:osf:socarx:42gcb_v1 is not listed on IDEAS
    19. Schwerdt, Guido & West, Martin R. & Winters, Marcus A., 2017. "The effects of test-based retention on student outcomes over time: Regression discontinuity evidence from Florida," Journal of Public Economics, Elsevier, vol. 152(C), pages 154-169.
    20. Simone Balestra & Uschi Backes-Gellner, 2014. "Heterogeneous effects of pupil-to-teacher ratio policies - A look at class size reduction and teacher aide," Economics of Education Working Paper Series 0102, University of Zurich, Department of Business Administration (IBW), revised Apr 2017.
    21. Caterina Santi & Pietro Santoleri, 2017. "Exploring the link between innovation and growth in Chilean firms," Small Business Economics, Springer, vol. 49(2), pages 445-467, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:68:y:2019:i:c:p:89-103. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.