IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18283.html

Can the Keyboard Beat the Good Old Pen? Computer-Based Testing and Students’ Performance

Author

Listed:
  • Freitas, Pedro

    (Nova School of Business and Economics)

  • Nunes, Luis Catela

    (Nova School of Business and Economics)

  • Reis, Ana B

    (Nova School of Business and Economics)

  • Pereira dos Santos, João

    (ISEG)

Abstract

Computer-based testing is increasingly being adopted by educational institutions worldwide. However, whether this transition from paper-based testing leads to different outcomes in student performance remains an open question. This paper assesses the impact of computer-based testing by examining a large-scale pilot programme for low-stakes exams implemented in Portugal in 2022. We leverage rich student-level data to implement pooled OLS and difference-in-differences approaches. Our results indicate that students who used computer-based testing performed worse than their peers using paper-based testing by 5 to 14 percentage points, on average. This negative effect is concentrated in specific question formats, namely questions requiring the analysis of figures. We discuss the implications of our findings for the large-scale implementation of computer-based testing.

Suggested Citation

  • Freitas, Pedro & Nunes, Luis Catela & Reis, Ana B & Pereira dos Santos, João, 2025. "Can the Keyboard Beat the Good Old Pen? Computer-Based Testing and Students’ Performance," IZA Discussion Papers 18283, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp18283
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18283.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    2. Hanho Jeong, 2014. "A comparative study of scores on computer-based tests and paper-based tests," Behaviour and Information Technology, Taylor & Francis Journals, vol. 33(4), pages 410-422, April.
    3. Wuthisatian, Rattaphon, 2020. "Student exam performance in different proctored environments: Evidence from an online economics course," International Review of Economics Education, Elsevier, vol. 35(C).
    4. Stephen Machin & Sandra McNally & Olmo Silva, 2007. "New Technology in Schools: Is There a Payoff?," Economic Journal, Royal Economic Society, vol. 117(522), pages 1145-1167, July.
    5. Julian Cristia & Pablo Ibarrarán & Santiago Cueto & Ana Santiago & Eugenio Severín, 2017. "Technology and Child Development: Evidence from the One Laptop per Child Program," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 295-320, July.
    6. Victor Lavy & Edith Sand & Moses Shayo, 2022. "Discrimination Between Religious and Non-Religious Groups: Evidence from Marking High-Stakes Exams," The Economic Journal, Royal Economic Society, vol. 132(646), pages 2308-2324.
    7. McElroy, Katherine, 2023. "Does test-based accountability improve more than just test scores?," Economics of Education Review, Elsevier, vol. 94(C).
    8. Agostinelli, Francesco & Doepke, Matthias & Sorrenti, Giuseppe & Zilibotti, Fabrizio, 2022. "When the great equalizer shuts down: Schools, peers, and parents in pandemic times," Journal of Public Economics, Elsevier, vol. 206(C).
    9. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    10. Azmat, Ghazala & Iriberri, Nagore, 2010. "The importance of relative performance feedback information: Evidence from a natural experiment using high school students," Journal of Public Economics, Elsevier, vol. 94(7-8), pages 435-452, August.
    11. William T. Alpert & Kenneth A. Couch & Oskar R. Harmon, 2016. "A Randomized Assessment of Online Learning," American Economic Review, American Economic Association, vol. 106(5), pages 378-382, May.
    12. Brian A. Jacob & Lars Lefgren, 2004. "Remedial Education and Student Achievement: A Regression-Discontinuity Analysis," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 226-244, February.
    13. Bacher-Hicks, Andrew & Goodman, Joshua & Mulhern, Christine, 2021. "Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time," Journal of Public Economics, Elsevier, vol. 193(C).
    14. Tzu-Hua Wang & Chien-Hui Kao & Hsiang-Chun Chen, 2021. "Factors Associated with the Equivalence of the Scores of Computer-Based Test and Paper-and-Pencil Test: Presentation Type, Item Difficulty and Administration Order," Sustainability, MDPI, vol. 13(17), pages 1-14, August.
    15. Hall, Caroline & Lundin, Martin & Sibbmark, Kristina, 2021. "A laptop for every child? The impact of technology on human capital formation," Labour Economics, Elsevier, vol. 69(C).
    16. Backes, Ben & Cowan, James, 2019. "Is the pen mightier than the keyboard? The effect of online testing on measured student achievement," Economics of Education Review, Elsevier, vol. 68(C), pages 89-103.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cardim, Joana & Molina-Millán, Teresa & Vicente, Pedro C., 2023. "Can technology improve the classroom experience in primary education? An African experiment on a worldwide program," Journal of Development Economics, Elsevier, vol. 164(C).
    2. Sanchis-Guarner, Rosa & Montalbán, José & Weinhardt, Felix, 2025. "Home broadband and human capital formation," Economics of Education Review, Elsevier, vol. 108(C).
    3. Beam, Emily A. & Mukherjee, Priya & Navarro-Sola, Laia, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," IZA Discussion Papers 15596, IZA Network @ LISER.
    4. Clark, Andrew E. & Nong, Huifu & Zhu, Hongjia & Zhu, Rong, 2021. "Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic," China Economic Review, Elsevier, vol. 68(C).
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Prashant Poddar & Valentina Rotondi & Ridhi Kashyap, 2025. "Access to technology and foundational math proficiency among students: empirical evidence from India," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-15, December.
    7. Liao, Haoye & Ma, Sen & Xue, Hao, 2022. "Does school shutdown increase inequality in academic performance? Evidence from COVID-19 pandemic in China," China Economic Review, Elsevier, vol. 75(C).
    8. Xiaoyue Shan & Ulf Zölitz & Uschi Backes-Gellner, 2025. "Disconnecting Women: Gender Disparities in the Impact of Online Instruction," CESifo Working Paper Series 11997, CESifo.
    9. Ainoa Aparicio Fenoll, 2025. "The different effects of Covid on children with parents in teleworkable and non-teleworkable occupations," Journal of Population Economics, Springer;European Society for Population Economics, vol. 38(1), pages 1-28, March.
    10. Yekaterina Chzhen & Jennifer Symonds & Dympna Devine & Júlia Mikolai & Susan Harkness & Seaneen Sloan & Gabriela Martinez Sainz, 2022. "Learning in a Pandemic: Primary School children’s Emotional Engagement with Remote Schooling during the spring 2020 Covid-19 Lockdown in Ireland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(4), pages 1517-1538, August.
    11. Harbatkin, Erica & Strunk, Katharine O. & McIlwain, Aliyah, 2023. "School turnaround in a pandemic: An examination of the outsized implications of COVID-19 on low-performing turnaround schools, districts, and their communities," Economics of Education Review, Elsevier, vol. 97(C).
    12. Hall, Caroline & Lundin, Martin & Sibbmark, Kristina, 2019. "A laptop for every child? The impact of ICT on educational outcomes," Working Paper Series 2019:26, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    13. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," CEPR Discussion Papers 15761, C.E.P.R. Discussion Papers.
    14. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    15. Bonaccolto-Töpfer, Marina & Castagnetti, Carolina, 2021. "The COVID-19 pandemic: A threat to higher education?," Discussion Papers 117, Friedrich-Alexander University Erlangen-Nuremberg, Chair of Labour and Regional Economics.
    16. Aparicio Fenoll, Ainoa, 2022. "The Uneven Effect of COVID School Closures: Parents in Teleworkable vs. Non-teleworkable Occupations," IZA Discussion Papers 15754, IZA Network @ LISER.
    17. Hideo Akabayashi & Shimpei Taguchi & Mirka Zvedelikova, 2023. "School ICT resources, teachers, and online education:Evidence from school closures in Japan during the COVID-19 pandemic," Keio-IES Discussion Paper Series 2023-008, Institute for Economics Studies, Keio University.
    18. Comi, Simona Lorena & Argentin, Gianluca & Gui, Marco & Origo, Federica & Pagani, Laura, 2017. "Is it the way they use it? Teachers, ICT and student achievement," Economics of Education Review, Elsevier, vol. 56(C), pages 24-39.
    19. Beland, Louis-Philippe & Murphy, Richard, 2016. "Ill Communication: Technology, distraction & student performance," Labour Economics, Elsevier, vol. 41(C), pages 61-76.
    20. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18283. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.