IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v112y2025ics0149718925001053.html

Direct measurement of learning outcomes in higher education: A proposal of nine standardized scales for continuous improvement in engineering programs

Author

Listed:
  • Hernández-Campos, Mónica
  • Prado-Calderón, Jorge Esteban
  • Gonzalez-Torres, Antonio
  • García-Peñalvo, Francisco José

Abstract

Despite the growing interest in measuring learning outcomes as evidence of comprehensive education in higher education institutions, objectively assessing these outcomes remains a challenge. This study addresses this problem by developing and validating nine standardized direct assessment scales tailored for engineering programs, with the goal of enhancing quality assurance. Study 1 focused on designing indicators for the scales through expert workshops and peer judgment processes, looking for content validity using Fleiss Kappa. Study 2 involved instructors using these instruments to assess various learning outcomes in a STEM-focused university. Analysis of 1420 assessments revealed strong validity and reliability indicators, including unifactorial structures, high reliability, discrimination, and appropriate difficulty levels for all scales. The research contributes evidence supporting the content validity of these assessment instruments, offering valuable tools for researchers and practitioners in engineering education to support continuous improvement. These scales have the potential to support future comparative research, inform quality assurance decisions, and enhance comprehensive education practices. Finally, the study discusses its limitations and provides directions for future research.

Suggested Citation

  • Hernández-Campos, Mónica & Prado-Calderón, Jorge Esteban & Gonzalez-Torres, Antonio & García-Peñalvo, Francisco José, 2025. "Direct measurement of learning outcomes in higher education: A proposal of nine standardized scales for continuous improvement in engineering programs," Evaluation and Program Planning, Elsevier, vol. 112(C).
    • Handle: RePEc:eee:epplan:v:112:y:2025:i:c:s0149718925001053
    DOI: 10.1016/j.evalprogplan.2025.102638
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718925001053
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2025.102638?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. William Black & Barry J. Babin, 2019. "Multivariate Data Analysis: Its Approach, Evolution, and Impact," Springer Books, in: Barry J. Babin & Marko Sarstedt (ed.), The Great Facilitator, pages 121-130, Springer.
    2. Deborah Nusche, 2008. "Assessment of Learning Outcomes in Higher Education: a comparative review of selected practices," OECD Education Working Papers 15, OECD Publishing.
    3. Saqib Saeed & Abdullah M. Almuhaideb & Yasser A. Bamarouf & Dina A. Alabaad & Hina Gull & Madeeha Saqib & Sardar Zafar Iqbal & Asiya Abdus Salam, 2021. "Sustainable Program Assessment Practices: A Review of the ABET and NCAAA Computer Information Systems Accreditation Process," IJERPH, MDPI, vol. 18(23), pages 1-14, December.
    4. Henry Kaiser, 1970. "A second generation little jiffy," Psychometrika, Springer;The Psychometric Society, vol. 35(4), pages 401-415, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yanqun Yang & Linwei Wang & Said M. Easa & Xinyi Zheng, 2022. "Analysis of Electric Bicycle Riders’ Use of Mobile Phones While Riding on Campus," IJERPH, MDPI, vol. 19(10), pages 1-15, May.
    2. Nur Hazira Abdul Halim & Saliza Sulaiman & Zuraidah Ismail & Shahariah Asmuni & Idaya Husna Mohd & Tuan Badrol Hisham Tuan Besar, 2026. "Digital Work Environment Innovation and Employee Turnover Intention in the Retail Industry: Evidence from the Eyewear Sector," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 10(1), pages 4787-4791, January.
    3. Xiaozhe Hui & Syed Hassan Raza & Sanan Waheed Khan & Umer Zaman & Emenyeonu C. Ogadimma, 2023. "Exploring Regenerative Tourism Using Media Richness Theory: Emerging Role of Immersive Journalism, Metaverse-Based Promotion, Eco-Literacy, and Pro-Environmental Behavior," Sustainability, MDPI, vol. 15(6), pages 1-20, March.
    4. Asai, Masayasu & Langer, Vibeke & Frederiksen, Pia & Jacobsen, Brian H., 2014. "Livestock farmer perceptions of successful collaborative arrangements for manure exchange: A study in Denmark," Agricultural Systems, Elsevier, vol. 128(C), pages 55-65.
    5. Garima & Atul Dhingra & Piera Centobelli & Roberto Cerchione, 2023. "Factors and Activities Considered by First Generation Agripreneurs for Agri-Business Sustainable Development: A Study of Haryana, India," Sustainability, MDPI, vol. 15(9), pages 1-18, April.
    6. Andrea C Vial & Janine Bosak & Patrick C Flood & John F Dovidio, 2021. "Individual variation in role construal predicts responses to third-party biases in hiring contexts," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-28, February.
    7. Siwarit Pongsakornrungsilp & Pimlapas Pongsakornrungsilp & Theeranuch Pusaksrikit & Pimmada Wichasin & Vikas Kumar, 2021. "Co-Creating a Sustainable Regional Brand from Multiple Sub-Brands: The Andaman Tourism Cluster of Thailand," Sustainability, MDPI, vol. 13(16), pages 1-23, August.
    8. Rehman, Anisur & Behera, Rajat Kumar & Islam, Md Saiful & Elahi, Yasir Arafat & Abbasi, Faraz Ahmad & Imtiaz, Asma, 2024. "Drivers of metaverse adoption for enhancing marketing capabilities of retail SMEs," Technology in Society, Elsevier, vol. 79(C).
    9. Admassu N. Lamu, 2020. "Does linear equating improve prediction in mapping? Crosswalking MacNew onto EQ-5D-5L value sets," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(6), pages 903-915, August.
    10. Nuanphromsakul, Kajohnjak & Szczepańska-Woszczyna, Katarzyna & Kot, Sebastian & Chaveesuk, Singha & Chaiyasoonthorn, Wornchanok, . "Sustainability of Rubber Farmers Cooperatives: Empirical Evaluation of Determining Factors," AGRIS on-line Papers in Economics and Informatics, Czech University of Life Sciences Prague, Faculty of Economics and Management, vol. 14(4).
    11. Bangyi Yan & Shiguang Ni & Xi Wang & Jin Liu & Qianjing Zhang & Kaiping Peng, 2020. "Using Virtual Reality to Validate the Chinese Version of the Independent Television Commission-Sense of Presence Inventory," SAGE Open, , vol. 10(2), pages 21582440209, May.
    12. Christoph, Inken B. & Roosen, Jutta & Bruhn, Maike, 2006. "Willingness to pay for genetically modified food and non-food products," 2006 Annual meeting, July 23-26, Long Beach, CA 21303, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    13. repec:plo:pone00:0039325 is not listed on IDEAS
    14. Ermos Michael Jama & Bupe Gertrude Mwanza & Erastus Mishengu Mwanaumo, 2024. "Strategies for E-Procurement Adoption by Small and Medium-sized Enterprises: Insights from South Sudan," African Journal of Commercial Studies, African Journal of Commercial Studies, vol. 4(3).
    15. Carolina Herrando & Julio Jiménez-Martínez & María José Martín-De Hoyos, 2017. "Passion at first sight: how to engage users in social commerce contexts," Electronic Commerce Research, Springer, vol. 17(4), pages 701-720, December.
    16. Gyensare, Michael Asiedu & Miri, Domnan & Zahoor, Nadia & Alajaty, Mahmoud, 2024. "Aspiring to go abroad: How and when international entrepreneurial aspiration fuel emerging markets entrepreneurial ventures’ internationalisation speed," Journal of International Management, Elsevier, vol. 30(2).
    17. Salmi Mohd Isa, PhD (Hull, UK), 2012. "An Analysis of Corporate Social Responsibility (CSR) on Stakeholders Loyalty: Perceptions of Malaysian Organizations," International Journal of Business and Social Research, MIR Center for Socio-Economic Research, vol. 2(7), pages 1-12, December.
    18. Hartieni, Patricia & Joewono, Tri Basuki & Dharmowijoyo, Dimas, 2024. "The effects of planned behaviour, spatiotemporal variables and lifestyle on public transport use: An exploratory study," Transportation Research Part A: Policy and Practice, Elsevier, vol. 190(C).
    19. Iyer, Pooja & Bright, Laura F., 2024. "Navigating a paradigm shift: Technology and user acceptance of big data and artificial intelligence among advertising and marketing practitioners," Journal of Business Research, Elsevier, vol. 180(C).
    20. Chetan Doddamani & M. Manoj, 2023. "Analysis of the influences of built environment measures on household car and motorcycle ownership decisions in Hubli-Dharwad cities," Transportation, Springer, vol. 50(1), pages 205-243, February.
    21. Nobes, Christopher & Stadler, Christian, 2013. "How arbitrary are international accounting classifications? Lessons from centuries of classifying in many disciplines, and experiments with IFRS data," Accounting, Organizations and Society, Elsevier, vol. 38(8), pages 573-595.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:112:y:2025:i:c:s0149718925001053. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.