IDEAS home Printed from https://ideas.repec.org/a/eee/teinso/v84y2026ics0160791x25002490.html

Development and validation of Generative AI Competence Scale (GenAIComp) among university students

Author

Listed:
  • Lee, Seul Chan
  • Baby, Tiju
  • Vongvit, Rattawut
  • Lee, Jieun
  • Kim, Young Woo
  • Cha, Min Chul
  • Yoon, Sol Hee

Abstract

The rapid development of Generative Artificial Intelligence (Generative AI) across several sectors underscores the need for a systematic tool to evaluate AI competence. Current digital literacy frameworks lack AI-specific competencies, resulting in inconsistencies in the assessment of AI competence. This study aims to establish a standardized assessment framework for Generative AI competence by identifying key skill factors and empirically validating a structured evaluation tool called the Generative AI Competence Scale (GenAIComp). The proposed GenAIComp has five essential factors: Information and Data Literacy, Communication and Collaboration, Digital Content Creation, Safety and Ethics, and Problem-Solving. A quantitative approach was employed, incorporating expert validation, pilot testing, and extensive empirical evaluation involving 1000 participants, principally university students. The factor analysis confirmed a robust 5-factor structure with strong psychometric properties. The final model demonstrated excellent fit indices, confirming its reliability and validity in assessing Generative AI competence across the five key factors. Research demonstrates that educational background considerably impacts AI competence, with individuals from technical disciplines showing a greater aptitude for problem-solving and content generation. Gender-based disparities were noted, with males achieving marginally higher scores in several factors, but with minimal effect sizes. Correlation analysis indicated that perceived AI expertise and frequency of AI utilization significantly influenced competence, especially in data literacy and problem-solving, and exhibited less correlation with ethical awareness. GenAIComp provides a reliable tool for assessing AI competence, helping educators, industry experts, and policymakers to design AI training programs and integrate AI literacy into curricula and thereby AI technology advancement in society. Future research should explore its applicability across cultures and include performance-based assessments to enhance AI competence.

Suggested Citation

  • Lee, Seul Chan & Baby, Tiju & Vongvit, Rattawut & Lee, Jieun & Kim, Young Woo & Cha, Min Chul & Yoon, Sol Hee, 2026. "Development and validation of Generative AI Competence Scale (GenAIComp) among university students," Technology in Society, Elsevier, vol. 84(C).
    • Handle: RePEc:eee:teinso:v:84:y:2026:i:c:s0160791x25002490
    DOI: 10.1016/j.techsoc.2025.103059
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0160791X25002490
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techsoc.2025.103059?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Lee Cronbach, 1951. "Coefficient alpha and the internal structure of tests," Psychometrika, Springer;The Psychometric Society, vol. 16(3), pages 297-334, September.
    2. Scuotto, Veronica & Tzanidis, Theofilos & Usai, Antonio & Quaglia, Roberto, 2023. "The digital humanism era triggered by individual creativity," Journal of Business Research, Elsevier, vol. 158(C).
    3. William Black & Barry J. Babin, 2019. "Multivariate Data Analysis: Its Approach, Evolution, and Impact," Springer Books, in: Barry J. Babin & Marko Sarstedt (ed.), The Great Facilitator, pages 121-130, Springer.
    4. Henry Kaiser, 1974. "An index of factorial simplicity," Psychometrika, Springer;The Psychometric Society, vol. 39(1), pages 31-36, March.
    5. Eszter Hargittai & Steven Shafer, 2006. "Differences in Actual and Perceived Online Skills: The Role of Gender," Social Science Quarterly, Southwestern Social Science Association, vol. 87(2), pages 432-448, June.
    6. Riina Vuorikari & Yves Punie & Stephanie Carretero Gomez & Godelieve Van Den Brande, 2016. "DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: the Conceptual Reference Model," JRC Research Reports JRC101254, Joint Research Centre.
    7. Hashmi, Nada & Bal, Anjali S., 2024. "Generative AI in higher education and beyond," Business Horizons, Elsevier, vol. 67(5), pages 607-614.
    8. Christine Redecker, 2017. "European Framework for the Digital Competence of Educators: DigCompEdu," JRC Research Reports JRC107466, Joint Research Centre.
    9. Kate Crawford & Ryan Calo, 2016. "There is a blind spot in AI research," Nature, Nature, vol. 538(7625), pages 311-313, October.
    10. VUORIKARI Riina & KLUZER Stefano & PUNIE Yves, 2022. "DigComp 2.2: The Digital Competence Framework for Citizens - With new examples of knowledge, skills and attitudes," JRC Research Reports JRC128415, Joint Research Centre.
    11. Alison Powell & Alissa Cooper, 2011. "Net neutrality discourses: comparing advocacy and regulatory arguments in the United States and the United Kingdom," LSE Research Online Documents on Economics 39024, London School of Economics and Political Science, LSE Library.
    12. Czakon, Wojciech & Klimas, Patrycja & Kawa, Arkadiusz & Kraus, Sascha, 2023. "How myopic are managers? Development and validation of a multidimensional strategic myopia scale," Journal of Business Research, Elsevier, vol. 157(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jayaram, Jayanth & Tan, Keah-Choon, 2010. "Supply chain integration with third-party logistics providers," International Journal of Production Economics, Elsevier, vol. 125(2), pages 262-271, June.
    2. Eunice Kabahinda & Rogers Mwesigwa, 2023. "Trust Mediates the Relationship Between Stakeholder Behavior and Stakeholder Management of Public Private Partnership Projects in Uganda," Public Organization Review, Springer, vol. 23(1), pages 245-263, March.
    3. Dyego Carlos Souza Anacleto de Araújo & Sylmara Nayara Pereira & Willian Melo dos Santos & Pedro Wlisses dos Santos Menezes & Kérilin Stancine dos Santos Rocha & Sabrina Cerqueira-Santos & André Faro , 2021. "Brazilian version of the Personal Report of Communication Apprehension: Cross-cultural adaptation and psychometric evaluation among healthcare students," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-13, February.
    4. Jaime Martín-Martín & Bella Pajares-Hachero & Emilio Alba-Conejo & Nuria Ribelles & Antonio I. Cuesta-Vargas & Cristina Roldán-Jiménez, 2023. "Validation of the Upper Limb Functional Index on Breast Cancer Survivor," IJERPH, MDPI, vol. 20(6), pages 1-10, March.
    5. Kanishka Gupta & Abdul Wajid & Dolly Gaur, 2024. "Determinants of continuous intention to use FinTech services: the moderating role of COVID-19," Journal of Financial Services Marketing, Palgrave Macmillan, vol. 29(2), pages 536-552, June.
    6. Nuno Garoupa & Rok Spruk, 2024. "Measuring Political Institutions in the Long Run: A Latent Variable Analysis of Political Regimes, 1810–2018," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 867-914, July.
    7. Hauck, Jana & Suess-Reyes, Julia & Beck, Susanne & Prügl, Reinhard & Frank, Hermann, 2016. "Measuring socioemotional wealth in family-owned and -managed firms: A validation and short form of the FIBER Scale," Journal of Family Business Strategy, Elsevier, vol. 7(3), pages 133-148.
    8. Hasnan Baber, 2020. "FinTech, Crowdfunding and Customer Retention in Islamic Banks," Vision, , vol. 24(3), pages 260-268, September.
    9. Fen Ren & Kexin Wang, 2022. "Modeling of the Chinese Dating App Use Motivation Scale According to Item Response Theory and Classical Test Theory," IJERPH, MDPI, vol. 19(21), pages 1-15, October.
    10. Ernest Bielinis & Jianzhong Xu & Aneta Anna Omelan, 2020. "A Novel Anti-Environmental Forest Experience Scale to Predict Preferred Pleasantness Associated with Forest Environments," IJERPH, MDPI, vol. 17(18), pages 1-18, September.
    11. Nguyen Ba Chien & Nguyen Nghi Thanh, 2022. "The Impact of Good Governance on the People’s Satisfaction with Public Administrative Services in Vietnam," Administrative Sciences, MDPI, vol. 12(1), pages 1-12, February.
    12. Raveenajit Kaur A. P. & Kalvant Singh & Alberto Luis August, 2021. "Exploring the Factor Structure of the Constructs of Technological, Pedagogical, and Content Knowledge (TPACK): An Exploratory Factor Analysis Based on the Perceptions of TESOL Pre-Service Teachers at a British Private University in Malaysia," Research Journal of Education, Academic Research Publishing Group, vol. 7(2), pages 103-115, 06-2021.
    13. Ricardo Figueiredo Belchior & Roisin Lyons, 2021. "Explaining entrepreneurial intentions, nascent entrepreneurial behavior and new business creation with social cognitive career theory – a 5-year longitudinal analysis," International Entrepreneurship and Management Journal, Springer, vol. 17(4), pages 1945-1972, December.
    14. Cemil Kuzey, 2018. "Impact of Health Care Employees Job Satisfaction on Organizational Performance Support Vector Machine Approach," Journal of Economics and Financial Analysis, Tripal Publishing House, vol. 2(1), pages 45-68.
    15. Katharina Fellnhofer & Kaisu Puumalainen & Helena Sjögrén, 0. "Entrepreneurial orientation in work groups – effects of individuals and group characteristics," International Entrepreneurship and Management Journal, Springer, vol. 0, pages 1-37.
    16. Jesús Antonio López Cabrera & Francisco Gabriel Villarreal & Diego Cardoso López, 2023. "Una propuesta de medición de la inclusión financiera en México," Remef - Revista Mexicana de Economía y Finanzas Nueva Época REMEF (The Mexican Journal of Economics and Finance), Instituto Mexicano de Ejecutivos de Finanzas, IMEF, vol. 18(3), pages 1-41, Julio - S.
    17. Silvia Mariela Méndez-Prado & Vanessa Rodriguez & Kevin Peralta-Rizzo & Patricia Everaert & Martin Valcke, 2023. "An Assessment Tool to Identify the Financial Literacy Level of Financial Education Programs Participants’ Executed by Ecuadorian Financial Institutions," Sustainability, MDPI, vol. 15(2), pages 1-24, January.
    18. Mayara Moraes Monteiro & João de Abreu e Silva & Jesper Bláfoss Ingvardson & Otto Anker Nielsen & Jorge Pinho de Sousa, 2021. "Public Transport Use and Satisfaction by International Students and Researchers," Sustainability, MDPI, vol. 13(15), pages 1-17, July.
    19. César Torres Martín & Christian Acal & Mohammed El Homrani & Ángel Custodio Mingorance Estrada, 2021. "Impact on the Virtual Learning Environment Due to COVID-19," Sustainability, MDPI, vol. 13(2), pages 1-16, January.
    20. Vieira, Kelmara Mendes & Rosenblum, Tamara Otilia Amaral & Matheis, Taiane Keila, 2022. "And tomorrow, how will it be? Developing a Financial Preparation for Retirement Scale (FPRS)," Journal of Behavioral and Experimental Finance, Elsevier, vol. 35(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:teinso:v:84:y:2026:i:c:s0160791x25002490. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/technology-in-society .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.