IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18283.html

Can the Keyboard Beat the Good Old Pen? Computer-Based Testing and Students’ Performance

Author

Listed:
  • Freitas, Pedro

    (University of Oxford)

  • Nunes, Luis Catela

    (Nova School of Business and Economics)

  • Reis, Ana B

    (Nova School of Business and Economics)

  • Pereira dos Santos, João

    (ISEG)

Abstract

Computer-based testing is increasingly being adopted by educational institutions worldwide. However, whether this transition from paper-based testing leads to different outcomes in student performance remains an open question. This paper assesses the impact of computer-based testing by examining a large-scale pilot programme for low-stakes exams implemented in Portugal in 2022. We leverage rich student-level data to implement pooled OLS and difference-in-differences approaches. Our results indicate that students who used computer-based testing performed worse than their peers using paper-based testing by 5 to 14 percentage points, on average. This negative effect is concentrated in specific question formats, namely questions requiring the analysis of figures. We discuss the implications of our findings for the large-scale implementation of computer-based testing.

Suggested Citation

  • Freitas, Pedro & Nunes, Luis Catela & Reis, Ana B & Pereira dos Santos, João, 2025. "Can the Keyboard Beat the Good Old Pen? Computer-Based Testing and Students’ Performance," IZA Discussion Papers 18283, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp18283
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18283.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hanho Jeong, 2014. "A comparative study of scores on computer-based tests and paper-based tests," Behaviour and Information Technology, Taylor & Francis Journals, vol. 33(4), pages 410-422, April.
    2. Stephen Machin & Sandra McNally & Olmo Silva, 2007. "New Technology in Schools: Is There a Payoff?," Economic Journal, Royal Economic Society, vol. 117(522), pages 1145-1167, July.
    3. Julian Cristia & Pablo Ibarrarán & Santiago Cueto & Ana Santiago & Eugenio Severín, 2017. "Technology and Child Development: Evidence from the One Laptop per Child Program," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 295-320, July.
    4. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    5. Azmat, Ghazala & Iriberri, Nagore, 2010. "The importance of relative performance feedback information: Evidence from a natural experiment using high school students," Journal of Public Economics, Elsevier, vol. 94(7-8), pages 435-452, August.
    6. Agostinelli, Francesco & Doepke, Matthias & Sorrenti, Giuseppe & Zilibotti, Fabrizio, 2022. "When the great equalizer shuts down: Schools, peers, and parents in pandemic times," Journal of Public Economics, Elsevier, vol. 206(C).
    7. William T. Alpert & Kenneth A. Couch & Oskar R. Harmon, 2016. "A Randomized Assessment of Online Learning," American Economic Review, American Economic Association, vol. 106(5), pages 378-382, May.
    8. Tzu-Hua Wang & Chien-Hui Kao & Hsiang-Chun Chen, 2021. "Factors Associated with the Equivalence of the Scores of Computer-Based Test and Paper-and-Pencil Test: Presentation Type, Item Difficulty and Administration Order," Sustainability, MDPI, vol. 13(17), pages 1-14, August.
    9. Hall, Caroline & Lundin, Martin & Sibbmark, Kristina, 2021. "A laptop for every child? The impact of technology on human capital formation," Labour Economics, Elsevier, vol. 69(C).
    10. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," IZA Discussion Papers 14094, IZA Network @ LISER.
    11. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    12. Wuthisatian, Rattaphon, 2020. "Student exam performance in different proctored environments: Evidence from an online economics course," International Review of Economics Education, Elsevier, vol. 35(C).
    13. Victor Lavy & Edith Sand & Moses Shayo, 2022. "Discrimination Between Religious and Non-Religious Groups: Evidence from Marking High-Stakes Exams," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2308-2324.
    14. McElroy, Katherine, 2023. "Does test-based accountability improve more than just test scores?," Economics of Education Review, Elsevier, vol. 94(C).
    15. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    16. Bacher-Hicks, Andrew & Goodman, Joshua & Mulhern, Christine, 2021. "Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time," Journal of Public Economics, Elsevier, vol. 193(C).
    17. Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    2. Cardim, Joana & Molina-Millán, Teresa & Vicente, Pedro C., 2023. "Can technology improve the classroom experience in primary education? An African experiment on a worldwide program," Journal of Development Economics, Elsevier, vol. 164(C).
    3. Sanchis-Guarner, Rosa & Montalbán, José & Weinhardt, Felix, 2025. "Home broadband and human capital formation," Economics of Education Review, Elsevier, vol. 108(C).
    4. Prashant Poddar & Valentina Rotondi & Ridhi Kashyap, 2025. "Access to technology and foundational math proficiency among students: empirical evidence from India," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-15, December.
    5. Shinsuke Asakawa & Fumio Ohtake, 2021. "Impact of Temporary School Closure Due to COVID-19 on the Academic Achievement of Elementary School Students," Discussion Papers in Economics and Business 21-14, Osaka University, Graduate School of Economics.
    6. Sakaue, Katsuki & Wokadala, James & Ogawa, Keiichi, 2023. "Effect of parental engagement on children’s home-based continued learning during COVID-19–induced school closures: Evidence from Uganda," International Journal of Educational Development, Elsevier, vol. 100(C).
    7. Clark, Andrew E. & Nong, Huifu & Zhu, Hongjia & Zhu, Rong, 2021. "Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic," China Economic Review, Elsevier, vol. 68(C).
    8. Josep Amer-Mestre & Alaitz Ayarza-Astigarraga & Marta C. Lopes, 2024. "E-learning engagement gap during school closures: differences by academic performance," Applied Economics, Taylor & Francis Journals, vol. 56(3), pages 337-359, January.
    9. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    10. Liao, Haoye & Ma, Sen & Xue, Hao, 2022. "Does school shutdown increase inequality in academic performance? Evidence from COVID-19 pandemic in China," China Economic Review, Elsevier, vol. 75(C).
    11. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2024. "Telementoring and Homeschooling During School Closures: a Randomised Experiment in Rural Bangladesh," The Economic Journal, Royal Economic Society, vol. 134(662), pages 2418-2438.
    12. Xiaoyue Shan & Ulf Zölitz & Uschi Backes-Gellner, 2025. "Disconnecting Women: Gender Disparities in the Impact of Online Instruction," CESifo Working Paper Series 11997, CESifo.
    13. Ainoa Aparicio Fenoll, 2025. "The different effects of Covid on children with parents in teleworkable and non-teleworkable occupations," Journal of Population Economics, Springer;European Society for Population Economics, vol. 38(1), pages 1-28, March.
    14. Svaleryd, Helena & Vlachos, Jonas, 2022. "COVID-19 and School Closures," GLO Discussion Paper Series 1008, Global Labor Organization (GLO).
    15. Yekaterina Chzhen & Jennifer Symonds & Dympna Devine & Júlia Mikolai & Susan Harkness & Seaneen Sloan & Gabriela Martinez Sainz, 2022. "Learning in a Pandemic: Primary School children’s Emotional Engagement with Remote Schooling during the spring 2020 Covid-19 Lockdown in Ireland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(4), pages 1517-1538, August.
    16. Harbatkin, Erica & Strunk, Katharine O. & McIlwain, Aliyah, 2023. "School turnaround in a pandemic: An examination of the outsized implications of COVID-19 on low-performing turnaround schools, districts, and their communities," Economics of Education Review, Elsevier, vol. 97(C).
    17. Bet, German & Cristia, Julián P. & Ibarrarán, Pablo, 2014. "The Effects of Shared School Technology Access on Students Digital Skills in Peru," IZA Discussion Papers 7954, IZA Network @ LISER.
    18. Hall, Caroline & Lundin, Martin & Sibbmark, Kristina, 2019. "A laptop for every child? The impact of ICT on educational outcomes," Working Paper Series 2019:26, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    19. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," CEPR Discussion Papers 15761, Centre for Economic Policy Research.
    20. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18283. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.