IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v12y2022i1p21582440211068492.html
   My bibliography  Save this article

How Reliable and Valid are the Evaluations of Digital Competence in Higher Education: A Systematic Mapping Study

Author

Listed:
  • Rafael Saltos-Rivas
  • Pavel Novoa-Hernández
  • Rocío Serrano Rodríguez

Abstract

Evaluating digital competencies has become a topic of growing interest in recent years. Although several reviews and studies have summarized the main elements of progress and shortcomings in this area, some issues are yet to be explored. Very little information is available about the ways of ensuring the validity and reliability of the instrument used. This study addresses this issue through systematic mapping covering the period from January 2015 to July 2020. Based on 88 primary studies, we conclude that a growing number of studies have emerged over the years; most of them are based on European university students in social science programs; the quality of the journals where they were published is low; and the instruments used are mostly questionnaires and ad-hoc surveys that measure the various dimensions of digital competence based on participants’ perceptions. Out of the eight possible types of quality assessment, studies commonly report only four (one in the case of reliability and three in the case of validity). More than 50% of the studies do not provide evidence of having assessed the reliability and validity at the same time. Although participant discipline resulted in a significantly association with the practice of reporting reliability and validity assessments, no specific dimension explained such association. A non-parametric multivariate analysis reveals, among other interesting patterns, that the practice of not reporting quality assessments is more present in studies where participants belong to the fields of Engineering and Technology, Health Sciences, and Humanities.

Suggested Citation

  • Rafael Saltos-Rivas & Pavel Novoa-Hernández & Rocío Serrano Rodríguez, 2022. "How Reliable and Valid are the Evaluations of Digital Competence in Higher Education: A Systematic Mapping Study," SAGE Open, , vol. 12(1), pages 21582440211, January.
    • Handle: RePEc:sae:sagope:v:12:y:2022:i:1:p:21582440211068492
    DOI: 10.1177/21582440211068492
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/21582440211068492
    Download Restriction: no
    File URL: https://libkey.io/10.1177/21582440211068492?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Wei-Yin Loh, 2014. "Fifty Years of Classification and Regression Trees," International Statistical Review, International Statistical Institute, vol. 82(3), pages 329-348, December.
    2. Stephanie Carretero & Riina Vuorikari & Yves Punie, 2017. "DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use," JRC Research Reports JRC106281, Joint Research Centre.
    3. Atanu Sengupta & Sanjoy De, 2020. "Review of Literature," India Studies in Business and Economics, in: Assessing Performance of Banks in India Fifty Years After Nationalization, chapter 0, pages 15-30, Springer.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Annika Kreuder & Ulrich Frick & Katrin Rakoczy & Sabine J. Schlittmeier, 2024. "Digital competence in adolescents and young adults: a critical analysis of concomitant variables, methodologies and intervention strategies," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-20, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Núñez-Canal, Margarita & de Obesso, Mª de las Mercedes & Pérez-Rivero, Carlos Alberto, 2022. "New challenges in higher education: A study of the digital competence of educators in Covid times," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    2. Cristina Blasi Casagran & Colleen Boland & Elena Sánchez-Montijano & Eva Vilà Sanchez, 2021. "The Role of Emerging Predictive IT Tools in Effective Migration Governance," Politics and Governance, Cogitatio Press, vol. 9(4), pages 133-145.
    3. He Tingting, 2021. "Comparing Money and Time Donation: What Do Experiments Tell Us?," Marketing of Scientific and Research Organizations, Sciendo, vol. 41(3), pages 65-94, September.
    4. Alberto Cerezo-Narváez & Andrés Pastor-Fernández & Manuel Otero-Mateo & Pablo Ballesteros-Pérez, 2022. "The Influence of Knowledge on Managing Risk for the Success in Complex Construction Projects: The IPMA Approach," Sustainability, MDPI, vol. 14(15), pages 1-30, August.
    5. Rafidah Md Noor & Nadia Bella Gustiani Rasyidi & Tarak Nandy & Raenu Kolandaisamy, 2020. "Campus Shuttle Bus Route Optimization Using Machine Learning Predictive Analysis: A Case Study," Sustainability, MDPI, vol. 13(1), pages 1-24, December.
    6. Dominika Ehrenbergerová & Martin Hodula & Zuzana Gric, 2022. "Does capital-based regulation affect bank pricing policy?," Journal of Regulatory Economics, Springer, vol. 61(2), pages 135-167, April.
    7. Mohammed Khaled Al-Hanawi & Rubayyat Hashmi & Sarh Almubark & Ameerah M. N. Qattan & Mohammad Habibullah Pulok, 2020. "Socioeconomic Inequalities in Uptake of Breast Cancer Screening among Saudi Women: A Cross-Sectional Analysis of a National Survey," IJERPH, MDPI, vol. 17(6), pages 1-13, March.
    8. Ortega, José Luis, 2021. "How do media mention research papers? Structural analysis of blogs and news networks using citation coupling," Journal of Informetrics, Elsevier, vol. 15(3).
    9. Richard Grieveson & Michael Landesmann & Isilda Mara, 2021. "Potential Mobility from Africa, Middle East and EU Neighbouring Countries to Europe," wiiw Working Papers 199, The Vienna Institute for International Economic Studies, wiiw.
    10. Pham, Hanh Song Thi & Petersen, Bent, 2021. "The bargaining power, value capture, and export performance of Vietnamese manufacturers in global value chains," International Business Review, Elsevier, vol. 30(6).
    11. Wafa Alwakid & Sebastian Aparicio & David Urbano, 2021. "The Influence of Green Entrepreneurship on Sustainable Development in Saudi Arabia: The Role of Formal Institutions," IJERPH, MDPI, vol. 18(10), pages 1-23, May.
    12. Gary Gereffi, 2020. "What does the COVID-19 pandemic teach us about global value chains? The case of medical supplies," Journal of International Business Policy, Palgrave Macmillan, vol. 3(3), pages 287-301, September.
    13. E. Denny, 2022. "Long-term Energy Cost Labelling for Appliances: Evidence from a Randomised Controlled Trial in Ireland," Journal of Consumer Policy, Springer, vol. 45(3), pages 369-409, September.
    14. Yan, Ran & Wang, Shuaian & Du, Yuquan, 2020. "Development of a two-stage ship fuel consumption prediction and reduction model for a dry bulk ship," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 138(C).
    15. Kentaka Aruga & Md. Monirul Islam & Yoshihiro Zenno & Arifa Jannat, 2022. "Developing Novel Technique for Investigating Guidelines and Frameworks: A Text Mining Comparison between International and Japanese Green Bonds," JRFM, MDPI, vol. 15(9), pages 1-17, August.
    16. Lenka Mynaříková & Lukáš Novotný, 2020. "Knowledge Society Failure? Barriers in the Use of ICTs and Further Teacher Education in the Czech Republic," Sustainability, MDPI, vol. 12(17), pages 1-19, August.
    17. M. del Mar Solà & A. de Ayala & I. Galarraga, 2021. "The Effect of Providing Monetary Information on Energy Savings for Household Appliances: A Field Trial in Spain," Journal of Consumer Policy, Springer, vol. 44(2), pages 279-310, June.
    18. Bárbara Rodríguez & Federico Paris-Garcia, 2022. "Influence of Dance Programmes on Gait Parameters and Physical Parameters of the Lower Body in Older People: A Systematic Review," IJERPH, MDPI, vol. 19(3), pages 1-17, January.
    19. Brian Kiprop Ngetich & Nuryakin & Ika Nurul Qamari, 2022. "Research trends of supply chain management practice before and during pandemic: A bibliometric analysis," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 11(2), pages 01-15, March.
    20. Favourate Y. Mpofu, 2022. "Taxing the Digital Economy through Consumption Taxes (VAT) in African Countries: Possibilities, Constraints and Implications," IJFS, MDPI, vol. 10(3), pages 1-21, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:12:y:2022:i:1:p:21582440211068492. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.