IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v57y2023i6d10.1007_s11135-023-01612-z.html
   My bibliography  Save this article

Statistical methods to estimate the impact of remote teaching on university students’ performance

Author

Listed:
  • Silvia Bacci

    (University of Florence (IT))

  • Bruno Bertaccini

    (University of Florence (IT))

  • Simone Del Sarto

    (Department of Political Science, University of Perugia (IT), Via Pascoli, 20, 06132)

  • Leonardo Grilli

    (University of Florence (IT))

  • Carla Rampichini

    (University of Florence (IT))

Abstract

The COVID-19 pandemic manifested around the World since February 2020, leading to disruptive effects on many aspects of people social life. The suspension of face-to-face teaching activities in schools and universities was the first containment measure adopted by the Governments to deal with the spread of the virus. Remote teaching has been the emergency solution implemented by schools and universities to limit the damages of schools and universities closure to students’ learning. In this contribution we intend to suggest to policy makers and researchers how to assess the impact of emergency policies on remote learning in academia by analysing students’ careers. In particular, we exploit the quasi-experimental setting arising from the sudden implementation of remote teaching in the second semester of academic year 2019/2020: we compare the performance of the cohort 2019/2020, which represents the treatment group, with the performance of the cohort 2018/2019, which represents the control group. We distinguish the impact of remote teaching at two levels: degree program and single courses within a degree program. We suggest to use Difference-In-Differences approach in the former case and multilevel modeling in the latter one. The proposal is illustrated analysing administrative data referred to freshmen of cohorts 2018/2019 and 2019/2020 for a sample of degree programs of the University of Florence (Italy).

Suggested Citation

  • Silvia Bacci & Bruno Bertaccini & Simone Del Sarto & Leonardo Grilli & Carla Rampichini, 2023. "Statistical methods to estimate the impact of remote teaching on university students’ performance," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(6), pages 5513-5531, December.
    • Handle: RePEc:spr:qualqt:v:57:y:2023:i:6:d:10.1007_s11135-023-01612-z
    DOI: 10.1007/s11135-023-01612-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-023-01612-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-023-01612-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lechner, Michael, 2011. "The Estimation of Causal Effects by Difference-in-Difference Methods," Foundations and Trends(R) in Econometrics, now publishers, vol. 4(3), pages 165-224, November.
    2. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    3. Aucejo, Esteban M. & French, Jacob & Ugalde Araya, Maria Paola & Zafar, Basit, 2020. "The impact of COVID-19 on student experiences and expectations: Evidence from a survey," Journal of Public Economics, Elsevier, vol. 191(C).
    4. Arturo Realyvásquez-Vargas & Aidé Aracely Maldonado-Macías & Karina Cecilia Arredondo-Soto & Yolanda Baez-Lopez & Teresa Carrillo-Gutiérrez & Guadalupe Hernández-Escobedo, 2020. "The Impact of Environmental Factors on Academic Performance of University Students Taking Online Classes during the COVID-19 Pandemic in Mexico," Sustainability, MDPI, vol. 12(21), pages 1-22, November.
    5. Tino Lesener & Leonard Santiago Pleiss & Burkhard Gusy & Christine Wolter, 2020. "The Study Demands-Resources Framework: An Empirical Introduction," IJERPH, MDPI, vol. 17(14), pages 1-13, July.
    6. Alan Agresti & Claudia Tarantola, 2018. "Simple ways to interpret effects in modeling ordinal categorical data," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 210-223, August.
    7. T Gonzalez & M A de la Rubia & K P Hincz & M Comas-Lopez & Laia Subirats & Santi Fort & G M Sacha, 2020. "Influence of COVID-19 confinement on students’ performance in higher education," PLOS ONE, Public Library of Science, vol. 15(10), pages 1-23, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Veronica Ballerini & Alice Dominici & Martina Francesca Ferracane & Fiammetta Menchetti & Silvia Noirjean, 2025. "Stimulating creativity and grit of high school students with creative STEM activities: an RCT with noncompliance," Quality & Quantity: International Journal of Methodology, Springer, vol. 59(1), pages 605-634, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Binelli, Chiara & Comi, Simona & Meschi, Elena & Pagani, Laura, 2024. "Every cloud has a silver lining: The role of study time and class recordings on university students’ performance during COVID-19," Journal of Economic Behavior & Organization, Elsevier, vol. 225(C), pages 305-328.
    2. Demirtaş, Burak Kağan & Türk, Umut, 2022. "Student performance under asynchronous and synchronous methods in distance education: A quasi-field experiment," International Review of Economics Education, Elsevier, vol. 41(C).
    3. Alaa El-Sakran & Reem Salman & Ayman Alzaatreh, 2022. "Impacts of Emergency Remote Teaching on College Students Amid COVID-19 in the UAE," IJERPH, MDPI, vol. 19(5), pages 1-21, March.
    4. Thompson, Paul N. & Ward, Jason, 2021. "Only a Matter of Time? The Role of Time in School on Four-Day School Week Achievement Impacts," IZA Discussion Papers 14461, Institute of Labor Economics (IZA).
    5. Laura Ihm & Han Zhang & Alexandra van Vijfeijken & Mark G. Waugh, 2021. "Impacts of the Covid‐19 pandemic on the health of university students," International Journal of Health Planning and Management, Wiley Blackwell, vol. 36(3), pages 618-627, May.
    6. Vida Navickiene & Valentina Dagiene & Egle Jasute & Rita Butkiene & Daina Gudoniene, 2021. "Pandemic-Induced Qualitative Changes in the Process of University Studies from the Perspective of University Authorities," Sustainability, MDPI, vol. 13(17), pages 1-16, September.
    7. Bandar N. Alarifi & Steve Song, 2024. "Online vs in-person learning in higher education: effects on student achievement and recommendations for leadership," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-8, December.
    8. Sally Diviner Yaa Adzaku & Hayford Daniel Adzakpa, 2022. "Impact of COVID-19 on the Nature of Academic Work in Colleges of Education in Ghana," Technium Social Sciences Journal, Technium Science, vol. 27(1), pages 117-130, January.
    9. Ilie, S. & Maragkou, K., 2024. "University admissions during a pandemic," Cambridge Working Papers in Economics 2458, Faculty of Economics, University of Cambridge.
    10. Mauricio Acuña & Roberto Alvarez & Cristobal Avarca, 2024. "The Heterogeneous Impact of COVID-19 on University Student's Academic Performance," Working Papers wp556, University of Chile, Department of Economics.
    11. Valentina Dagiene & Egle Jasute & Vida Navickiene & Rita Butkiene & Daina Gudoniene, 2022. "Opportunities, Quality Factors, and Required Changes during the Pandemic Based on Higher Education Leaders’ Perspective," Sustainability, MDPI, vol. 14(3), pages 1-18, February.
    12. Pertold-Gebicka, Barbara, 2024. "Medium-run effects of COVID-19 induced distant learning on students’ academic performance," Labour Economics, Elsevier, vol. 89(C).
    13. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    14. Alex Hollingsworth & Krzysztof Karbownik & Melissa A. Thomasson & Anthony Wray, 2024. "The Gift of a Lifetime: The Hospital, Modern Medicine, and Mortality," American Economic Review, American Economic Association, vol. 114(7), pages 2201-2238, July.
    15. Teruaki Kido & Yuko Yotsumoto & Masamichi J. Hayashi, 2025. "Hierarchical representations of relative numerical magnitudes in the human frontoparietal cortex," Nature Communications, Nature, vol. 16(1), pages 1-15, December.
    16. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    17. Christopher Hassall & Michael Nisbet & Evan Norcliffe & He Wang, 2024. "The Potential Health Benefits of Urban Tree Planting Suggested through Immersive Environments," Land, MDPI, vol. 13(3), pages 1-12, February.
    18. Fei Li & Christopher Kajetan Wyczalkowski, 2023. "How buses alleviate unemployment and poverty: Lessons from a natural experiment in Clayton County, GA," Urban Studies, Urban Studies Journal Limited, vol. 60(13), pages 2632-2650, October.
    19. Jie Zhao & Ji Chen & Damien Beillouin & Hans Lambers & Yadong Yang & Pete Smith & Zhaohai Zeng & Jørgen E. Olesen & Huadong Zang, 2022. "Global systematic review with meta-analysis reveals yield advantage of legume-based rotations and its drivers," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    20. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:57:y:2023:i:6:d:10.1007_s11135-023-01612-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.