IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v112y2025ics0149718925001053.html

Direct measurement of learning outcomes in higher education: A proposal of nine standardized scales for continuous improvement in engineering programs

Author

Listed:
  • Hernández-Campos, Mónica
  • Prado-Calderón, Jorge Esteban
  • Gonzalez-Torres, Antonio
  • García-Peñalvo, Francisco José

Abstract

Despite the growing interest in measuring learning outcomes as evidence of comprehensive education in higher education institutions, objectively assessing these outcomes remains a challenge. This study addresses this problem by developing and validating nine standardized direct assessment scales tailored for engineering programs, with the goal of enhancing quality assurance. Study 1 focused on designing indicators for the scales through expert workshops and peer judgment processes, looking for content validity using Fleiss Kappa. Study 2 involved instructors using these instruments to assess various learning outcomes in a STEM-focused university. Analysis of 1420 assessments revealed strong validity and reliability indicators, including unifactorial structures, high reliability, discrimination, and appropriate difficulty levels for all scales. The research contributes evidence supporting the content validity of these assessment instruments, offering valuable tools for researchers and practitioners in engineering education to support continuous improvement. These scales have the potential to support future comparative research, inform quality assurance decisions, and enhance comprehensive education practices. Finally, the study discusses its limitations and provides directions for future research.

Suggested Citation

  • Hernández-Campos, Mónica & Prado-Calderón, Jorge Esteban & Gonzalez-Torres, Antonio & García-Peñalvo, Francisco José, 2025. "Direct measurement of learning outcomes in higher education: A proposal of nine standardized scales for continuous improvement in engineering programs," Evaluation and Program Planning, Elsevier, vol. 112(C).
    • Handle: RePEc:eee:epplan:v:112:y:2025:i:c:s0149718925001053
    DOI: 10.1016/j.evalprogplan.2025.102638
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718925001053
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2025.102638?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Deborah Nusche, 2008. "Assessment of Learning Outcomes in Higher Education: a comparative review of selected practices," OECD Education Working Papers 15, OECD Publishing.
    2. Saqib Saeed & Abdullah M. Almuhaideb & Yasser A. Bamarouf & Dina A. Alabaad & Hina Gull & Madeeha Saqib & Sardar Zafar Iqbal & Asiya Abdus Salam, 2021. "Sustainable Program Assessment Practices: A Review of the ABET and NCAAA Computer Information Systems Accreditation Process," IJERPH, MDPI, vol. 18(23), pages 1-14, December.
    3. Henry Kaiser, 1970. "A second generation little jiffy," Psychometrika, Springer;The Psychometric Society, vol. 35(4), pages 401-415, December.
    4. William Black & Barry J. Babin, 2019. "Multivariate Data Analysis: Its Approach, Evolution, and Impact," Springer Books, in: Barry J. Babin & Marko Sarstedt (ed.), The Great Facilitator, pages 121-130, Springer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yanqun Yang & Linwei Wang & Said M. Easa & Xinyi Zheng, 2022. "Analysis of Electric Bicycle Riders’ Use of Mobile Phones While Riding on Campus," IJERPH, MDPI, vol. 19(10), pages 1-15, May.
    2. Andrea C Vial & Janine Bosak & Patrick C Flood & John F Dovidio, 2021. "Individual variation in role construal predicts responses to third-party biases in hiring contexts," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-28, February.
    3. Siwarit Pongsakornrungsilp & Pimlapas Pongsakornrungsilp & Theeranuch Pusaksrikit & Pimmada Wichasin & Vikas Kumar, 2021. "Co-Creating a Sustainable Regional Brand from Multiple Sub-Brands: The Andaman Tourism Cluster of Thailand," Sustainability, MDPI, vol. 13(16), pages 1-23, August.
    4. Rehman, Anisur & Behera, Rajat Kumar & Islam, Md Saiful & Elahi, Yasir Arafat & Abbasi, Faraz Ahmad & Imtiaz, Asma, 2024. "Drivers of metaverse adoption for enhancing marketing capabilities of retail SMEs," Technology in Society, Elsevier, vol. 79(C).
    5. Nuanphromsakul, Kajohnjak & Szczepańska-Woszczyna, Katarzyna & Kot, Sebastian & Chaveesuk, Singha & Chaiyasoonthorn, Wornchanok, . "Sustainability of Rubber Farmers Cooperatives: Empirical Evaluation of Determining Factors," AGRIS on-line Papers in Economics and Informatics, Czech University of Life Sciences Prague, Faculty of Economics and Management, vol. 14(4).
    6. Bangyi Yan & Shiguang Ni & Xi Wang & Jin Liu & Qianjing Zhang & Kaiping Peng, 2020. "Using Virtual Reality to Validate the Chinese Version of the Independent Television Commission-Sense of Presence Inventory," SAGE Open, , vol. 10(2), pages 21582440209, May.
    7. Christoph, Inken B. & Roosen, Jutta & Bruhn, Maike, 2006. "Willingness to pay for genetically modified food and non-food products," 2006 Annual meeting, July 23-26, Long Beach, CA 21303, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    8. Iyer, Pooja & Bright, Laura F., 2024. "Navigating a paradigm shift: Technology and user acceptance of big data and artificial intelligence among advertising and marketing practitioners," Journal of Business Research, Elsevier, vol. 180(C).
    9. Chetan Doddamani & M. Manoj, 2023. "Analysis of the influences of built environment measures on household car and motorcycle ownership decisions in Hubli-Dharwad cities," Transportation, Springer, vol. 50(1), pages 205-243, February.
    10. Mahajan, Advitya Indu & Taggar, Rashi & Gupta, Suruchi, 2025. "Augmented reality in E-Commerce: A dual value and barrier-based perspective on Gen Z's satisfaction and AR-Integrated shopping behavior," Journal of Retailing and Consumer Services, Elsevier, vol. 87(C).
    11. Geemol John & Mohammad Amin Wani, 2025. "Psychometric Development of a Midlife Crisis Questionnaire (MLC-Q): A Dual Approach Using Exploratory and Confirmatory Factor Analysis," SAGE Open, , vol. 15(3), pages 21582440251, September.
    12. Marcin Chlebus, 2014. "One-day prediction of state of turbulence for financial instrument based on models for binary dependent variable," Ekonomia journal, Faculty of Economic Sciences, University of Warsaw, vol. 37.
    13. Al-Shaer, Habiba & Uyar, Ali & Kuzey, Cemil & Karaman, Abdullah S., 2023. "Do shareholders punish or reward excessive CSR engagement? Moderating effect of cash flow and firm growth," International Review of Financial Analysis, Elsevier, vol. 88(C).
    14. Faheem Ahmed & Luiz Fernando Capretz, 2011. "A business maturity model of software product line engineering," Information Systems Frontiers, Springer, vol. 13(4), pages 543-560, September.
    15. Sk. Faijan Bin Halim & Tasnim Murad Mamun & Jannatul Naim, 2026. "Economics of Farmers’ Willingness to Participate in Weather-Risk-Induced Crop Insurance in the Southwest Region of Bangladesh," Economics of Disasters and Climate Change, Springer, vol. 10(1), pages 1-24, April.
    16. Libin Yang & William Rea & Alethea Rea, 2015. "How much diversification potential is there in a single market? Evidence from the Australian Stock Exchange," Working Papers in Economics 15/07, University of Canterbury, Department of Economics and Finance.
    17. Ben-Shahar, Danny & Golan, Roni, 2014. "Real estate and personality," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 53(C), pages 111-119.
    18. Khalil Nimer & Muath Abdelqader & Cemil Kuzey & Ali Uyar, 2024. "Emission targeting and carbon emissions: The moderating effect of female directors," Business Strategy and the Environment, Wiley Blackwell, vol. 33(4), pages 3480-3504, May.
    19. Cortés-Sánchez, Julián David & Grueso, Merlin Patricia, 2017. "Factor analysis evaluation of Schein's career orientation inventory in Colombia," OSF Preprints jf5nq, Center for Open Science.
    20. Md. Mosharraf Hossain & Md. Kamrozzaman & Md. Mehedi Hasan Manik & Rahima Akther & Atiqur Rahaman & Md Sahab Uddin, 2024. "Antecedents of Job Satisfaction among Higher Secondary Teachers: Moderating Role of Training," International Journal of Science and Business, IJSAB International, vol. 42(1), pages 52-70.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:112:y:2025:i:c:s0149718925001053. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.