IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i6p3392-d770684.html
   My bibliography  Save this article

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Author

Listed:
  • Juan Bartolomé

    (TECNALIA, Basque Research and Technology Alliance (BRTA), 48160 Derio Bizkaia, Spain)

  • Pablo Garaizar

    (Engineering Faculty, University of Deusto, 48007 Bilbao, Spain)

Abstract

Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.

Suggested Citation

  • Juan Bartolomé & Pablo Garaizar, 2022. "Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations," Sustainability, MDPI, vol. 14(6), pages 1-28, March.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3392-:d:770684
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/6/3392/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/6/3392/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rafael Saltos-Rivas & Pavel Novoa-Hernández & Rocío Serrano Rodríguez, 2021. "On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-27, September.
    2. Rotimi Abidoye & Benson Teck Heng Lim & Yu-Cheng Lin & Junge Ma, 2022. "Equipping Property Graduates for the Digital Age," Sustainability, MDPI, vol. 14(2), pages 1-23, January.
    3. Stefano Kluzer & Laia Pujol Priego, 2018. "DigComp into action: Get inspired, make it happen. A user guide to the European Digital Competence Framework," JRC Research Reports JRC110624, Joint Research Centre.
    4. Ana Isabel Santos & Sandro Serpa, 2017. "The Importance of Promoting Digital Literacy in Higher Education," International Journal of Social Science Studies, Redfame publishing, vol. 5(6), pages 90-93, June.
    5. Kopecky, Kamil & Szotkowski, Rene & Aznar-Díaz, Inmaculada & Romero-Rodríguez, José-María, 2020. "The phenomenon of sharenting and its risks in the online environment. Experiences from Czech Republic and Spain," Children and Youth Services Review, Elsevier, vol. 110(C).
    6. Stephanie Carretero & Riina Vuorikari & Yves Punie, 2017. "DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use," JRC Research Reports JRC106281, Joint Research Centre.
    7. David Thissen, 1982. "Marginal maximum likelihood estimation for the one-parameter logistic model," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 175-186, June.
    8. Javier Portillo & Urtza Garay & Eneko Tejada & Naiara Bilbao, 2020. "Self-Perception of the Digital Competence of Educators during the COVID-19 Pandemic: A Cross-Analysis of Different Educational Stages," Sustainability, MDPI, vol. 12(23), pages 1-13, December.
    9. Ana Pérez-Escoda & Luis Miguel Pedrero-Esteban & Juana Rubio-Romero & Carlos Jiménez-Narros, 2021. "Fake News Reaching Young People on Social Networks: Distrust Challenging Media Literacy," Publications, MDPI, vol. 9(2), pages 1-16, June.
    10. Alexander J.A.M. Van Deursen & Ellen J. Helsper & Rebecca Eynon, 2016. "Development and validation of the Internet Skills Scale (ISS)," LSE Research Online Documents on Economics 64485, London School of Economics and Political Science, LSE Library.
    11. Katriona O’Sullivan & Serena Clark & Kevin Marshall & Malcolm MacLachlan, 2021. "A Just Digital framework to ensure equitable achievement of the Sustainable Development Goals," Nature Communications, Nature, vol. 12(1), pages 1-4, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yu Zhao & María Cruz Sánchez Gómez & Ana María Pinto Llorente & Liping Zhao, 2021. "Digital Competence in Higher Education: Students’ Perception and Personal Factors," Sustainability, MDPI, vol. 13(21), pages 1-17, November.
    2. Núñez-Canal, Margarita & de Obesso, Mª de las Mercedes & Pérez-Rivero, Carlos Alberto, 2022. "New challenges in higher education: A study of the digital competence of educators in Covid times," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    3. Junfeng Yang & Ahmed Tlili & Ronghuai Huang & Rongxia Zhuang & Kaushal Kumar Bhagat, 2021. "Development and Validation of a Digital Learning Competence Scale: A Comprehensive Review," Sustainability, MDPI, vol. 13(10), pages 1-14, May.
    4. Maria José Sá & Ana Isabel Santos & Sandro Serpa & Carlos Miguel Ferreira, 2021. "Digitainability—Digital Competences Post-COVID-19 for a Sustainable Society," Sustainability, MDPI, vol. 13(17), pages 1-22, August.
    5. Alberto Maydeu-Olivares & Rosa Montaño, 2013. "How Should We Assess the Fit of Rasch-Type Models? Approximating the Power of Goodness-of-Fit Statistics in Categorical Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 78(1), pages 116-133, January.
    6. Trung Tran & Manh-Toan Ho & Thanh-Hang Pham & Minh-Hoang Nguyen & Khanh-Linh P. Nguyen & Thu-Trang Vuong & Thanh-Huyen T. Nguyen & Thanh-Dung Nguyen & Thi-Linh Nguyen & Quy Khuc & Viet-Phuong La & Qua, 2020. "How Digital Natives Learn and Thrive in the Digital Age: Evidence from an Emerging Economy," Sustainability, MDPI, vol. 12(9), pages 1-24, May.
    7. Haris Krijestorac & Rajiv Garg & Prabhudev Konana, 2021. "Decisions Under the Illusion of Objectivity: Digital Embeddedness and B2B Purchasing," Production and Operations Management, Production and Operations Management Society, vol. 30(7), pages 2232-2251, July.
    8. Cees Glas, 1988. "The derivation of some tests for the rasch model from the multinomial distribution," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 525-546, December.
    9. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    10. Lamberti, Giuseppe & Lopez-Sintas, Jordi & Sukphan, Jakkapong, 2021. "The social process of internet appropriation: Living in a digitally advanced country benefits less well-educated Europeans," Telecommunications Policy, Elsevier, vol. 45(1).
    11. Artur Santoalha & Davide Consoli & Fulvio Castellacci, 2019. "Do digital skills foster green diversification? A study of European regions," Working Papers on Innovation Studies 20191029, Centre for Technology, Innovation and Culture, University of Oslo.
    12. Li Cai, 2010. "A Two-Tier Full-Information Item Factor Analysis Model with Applications," Psychometrika, Springer;The Psychometric Society, vol. 75(4), pages 581-612, December.
    13. David Thissen & Howard Wainer, 1982. "Some standard errors in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 47(4), pages 397-412, December.
    14. César Torres Martín & Christian Acal & Mohammed El Homrani & Ángel Custodio Mingorance Estrada, 2021. "Impact on the Virtual Learning Environment Due to COVID-19," Sustainability, MDPI, vol. 13(2), pages 1-16, January.
    15. Godbersen, Hendrik & Hausinger, Lena, 2022. "Die Travel Journey von Digital Natives und Digital Immigrants: Eine vergleichende Analyse der digitalen und analogen Customer Touchpoints im Buchungsprozess von Urlaubsreisen," KCMS Schriftenreihe, FOM Hochschule für Oekonomie & Management, KCMS KompetenzCentrum für Marketing & Sales Management, volume 6, number 6 edited by FOM Hochschule für Oekonomie & Management, KCMS KompetenzCentrum für Marketing & Sales Management.
    16. Romero-Rodríguez, José-M. & Kopecký, Kamil & García-González, Abel & Gómez-García, Gerardo, 2022. "Sharing images or videos of minors online: Validation of the Sharenting Evaluation Scale (SES)," Children and Youth Services Review, Elsevier, vol. 136(C).
    17. Sandro Serpa & Ana Isabel Santos & Carlos Miguel Ferreira, 2020. "Contributions of Ivan Illich to Education in a Digital Society," Academic Journal of Interdisciplinary Studies, Richtmann Publishing Ltd, vol. 9, March.
    18. Lorena Martín-Párraga & Carmen Llorente-Cejudo & Julio Barroso-Osuna, 2023. "Self-Perception of Digital Competence in University Lecturers: A Comparative Study between Universities in Spain and Peru According to the DigCompEdu Model," Societies, MDPI, vol. 13(6), pages 1-14, June.
    19. Carlos Miguel Ferreira & Sandro Serpa, 2017. "Challenges in the Teaching of Sociology in Higher Education. Contributions to a Discussion," Societies, MDPI, vol. 7(4), pages 1-11, October.
    20. German Ben-Hayun, Shiran & Perry-Hazan, Lotem, 2023. "In the same boat: Parents’ and teachers’ role in protecting elementary school students’ online rights," Children and Youth Services Review, Elsevier, vol. 146(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3392-:d:770684. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.