IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i8d10.1007_s11192-022-04436-0.html
   My bibliography  Save this article

Combining reference point based composite indicators with data envelopment analysis: application to the assessment of universities

Author

Listed:
  • Samira El Gibari

    (Universidad de Málaga
    University of Malaga)

  • Trinidad Gómez

    (University of Malaga)

  • Francisco Ruiz

    (University of Malaga)

Abstract

In our knowledge society, where universities are key players, the assessment of higher education institutions should meet the new demands of the present complex environment. This calls for the use of techniques that are able to manage this complexity. In this paper, we propose a novel combination of methodologies, jointly using a multi-criteria reference point scheme and the data envelopment analysis (DEA) for the assessment of universities. This combination allows us to take into account all the aspects regarded as relevant to assess university performance, and use them as outputs in the efficiency analysis. Our findings highlight the convenience to assess the university performance by using both compensatory and non-compensatory schemes. This way, the information provided allows to detect the actions needed to improve the performances of the universities, rather than just giving an overall performance measure. Furthermore, combining the use of composite indicators with the DEA analysis provides a more complete picture of the institutions assessed, allowing universities to check their efficiency and to detect their weaknesses and strengths accordingly. The approach is illustrated using data of 47 Spanish public universities for the academic year, 2016–2017.

Suggested Citation

  • Samira El Gibari & Trinidad Gómez & Francisco Ruiz, 2022. "Combining reference point based composite indicators with data envelopment analysis: application to the assessment of universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4363-4395, August.
    • Handle: RePEc:spr:scient:v:127:y:2022:i:8:d:10.1007_s11192-022-04436-0
    DOI: 10.1007/s11192-022-04436-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04436-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04436-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tommaso Agasisti & Cristian Barra & Roberto Zotti, 2019. "Research, knowledge transfer, and innovation: The effect of Italian universities’ efficiency on local economic development 2006−2012," Journal of Regional Science, Wiley Blackwell, vol. 59(5), pages 819-849, November.
    2. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    3. Ruiz, Francisco & El Gibari, Samira & Cabello, José M. & Gómez, Trinidad, 2020. "MRP-WSCI: Multiple reference point based weak and strong composite indicators," Omega, Elsevier, vol. 95(C).
    4. Bao Hoang Nguyen & Valentin Zelenyuk, 2021. "Aggregation of Outputs and Inputs for DEA Analysis of Hospital Efficiency: Economics, Operations Research and Data Science Perspectives," International Series in Operations Research & Management Science, in: Joe Zhu & Vincent Charles (ed.), Data-Enabled Analytics, pages 123-158, Springer.
    5. Adler, Nicole & Yazhemsky, Ekaterina, 2010. "Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction," European Journal of Operational Research, Elsevier, vol. 202(1), pages 273-284, April.
    6. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    7. Yang, Guo-liang & Fukuyama, Hirofumi & Song, Yao-yao, 2018. "Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model," Journal of Informetrics, Elsevier, vol. 12(1), pages 10-30.
    8. Martínez-Campillo, Almudena & Fernández-Santos, Yolanda, 2020. "The impact of the economic crisis on the (in)efficiency of public Higher Education institutions in Southern Europe: The case of Spanish universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    9. Daniel Torres-Salinas & Jose G. Moreno-Torres & Emilio Delgado-López-Cózar & Francisco Herrera, 2011. "A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ 2 A index," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 771-786, September.
    10. F Ruiz & J M Cabello & M Luque, 2011. "An application of reference point techniques to the calculation of synthetic sustainability indicators," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(1), pages 189-197, January.
    11. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    12. Eva M. de la Torre & Tommaso Agasisti & Carmen Perez-Esparrells, 2017. "The relevance of knowledge transfer for universities’ efficiency scores: an empirical approximation on the Spanish public higher education system," Research Evaluation, Oxford University Press, vol. 26(3), pages 211-229.
    13. Dyson, R. G. & Allen, R. & Camanho, A. S. & Podinovski, V. V. & Sarrico, C. S. & Shale, E. A., 2001. "Pitfalls and protocols in DEA," European Journal of Operational Research, Elsevier, vol. 132(2), pages 245-259, July.
    14. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    15. Pilar Murias & José Miguel & David Rodríguez, 2008. "A Composite Indicator for University Quality Assesment: The Case of Spanish Higher Education System," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 89(1), pages 129-146, October.
    16. Meng, Wei & Zhang, Daqun & Qi, Li & Liu, Wenbin, 2008. "Two-level DEA approaches in research evaluation," Omega, Elsevier, vol. 36(6), pages 950-957, December.
    17. Charles, Vincent & Aparicio, Juan & Zhu, Joe, 2019. "The curse of dimensionality of decision-making units: A simple approach to increase the discriminatory power of data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 279(3), pages 929-940.
    18. Zhong, Wei & Yuan, Wei & Li, Susan X. & Huang, Zhimin, 2011. "The performance evaluation of regional R&D investments in China: An application of DEA based on the first official China economic census data," Omega, Elsevier, vol. 39(4), pages 447-455, August.
    19. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    20. Blancas, Francisco J. & Caballero, Rafael & González, Mercedes & Lozano-Oyola, Macarena & Pérez, Fátima, 2010. "Goal programming synthetic indicators: An application for sustainable tourism in Andalusian coastal counties," Ecological Economics, Elsevier, vol. 69(11), pages 2158-2172, September.
    21. A. Charnes & W. W. Cooper & E. Rhodes, 1981. "Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through," Management Science, INFORMS, vol. 27(6), pages 668-697, June.
    22. Delimiro Visbal-Cadavid & Mónica Martínez-Gómez & Francisco Guijarro, 2017. "Assessing the Efficiency of Public Universities through DEA. A Case Study," Sustainability, MDPI, vol. 9(8), pages 1-19, August.
    23. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    24. Samira El Gibari & Trinidad Gómez & Francisco Ruiz, 2019. "Building composite indicators using multicriteria methods: a review," Journal of Business Economics, Springer, vol. 89(1), pages 1-24, February.
    25. Berbegal-Mirabent, Jasmina & Lafuente, Esteban & Solé, Francesc, 2013. "The pursuit of knowledge transfer activities: An efficiency analysis of Spanish universities," Journal of Business Research, Elsevier, vol. 66(10), pages 2051-2059.
    26. El Gibari, Samira & Gómez, Trinidad & Ruiz, Francisco, 2018. "Evaluating university performance using reference point based composite indicators," Journal of Informetrics, Elsevier, vol. 12(4), pages 1235-1250.
    27. Oğuzhan Alaşehir & Murat Perit Çakır & Cengiz Acartürk & Nazife Baykal & Ural Akbulut, 2014. "URAP-TR: a national ranking for Turkish universities based on academic performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 159-178, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zihong Liu & Haitao Xiong & Ying Sun, 2023. "Will Online MOOCs Improve the Efficiency of Chinese Higher Education Institutions? An Empirical Study Based on DEA," Sustainability, MDPI, vol. 15(7), pages 1-21, March.
    2. Francesco Chelli & Mariateresa Ciommi & Francesca Mariani & Gloria Polinesi & Maria Cristina Recchioni & Giuseppe Ricciardo Lamonica & Luca Salvati, 2022. "A Story of Strengths and Weaknesses in Tertiary Education: Evaluating ‘Mobility’ and ‘Opportunities’ in OECD Countries with Composite Indicators," Sustainability, MDPI, vol. 14(24), pages 1-19, December.
    3. Yan Xia & Jianxin You & Xiumeng Feng & Yingjie Xu & Hui Feng, 2023. "Clustering Analysis of Classified Performance Evaluation of Higher Education in Shanghai Based on Topsis Model," Sustainability, MDPI, vol. 15(8), pages 1-18, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shih-Pin Chen & Chung-Wei Chang, 2021. "Measuring the efficiency of university departments: an empirical study using data envelopment analysis and cluster analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5263-5284, June.
    2. Aparicio, Juan & López-Torres, Laura & Santín, Daniel, 2018. "Economic crisis and public education. A productivity analysis using a Hicks-Moorsteen index," Economic Modelling, Elsevier, vol. 71(C), pages 34-44.
    3. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    4. Samira El Gibari & Carmen Perez-Esparrells & Trinidad Gomez & Francisco Ruiz, 2021. "Analyzing the Impact of Spanish University Funding Policies on the Evolution of Their Performance: A Multi-Criteria Approach," Mathematics, MDPI, vol. 9(14), pages 1-34, July.
    5. Joan Crespo & Jesús Peiró-Palomino & Emili Tortosa-Ausina, 2020. "Does university performance have an economic payoff for their home regions? Evidence for the Spanish provinces," Working Papers 2020/20, Economics Department, Universitat Jaume I, Castellón (Spain).
    6. Tavares, Rafael Santos & Angulo-Meza, Lidia & Sant'Anna, Annibal Parracho, 2021. "A proposed multistage evaluation approach for Higher Education Institutions based on network Data envelopment analysis: A Brazilian experience," Evaluation and Program Planning, Elsevier, vol. 89(C).
    7. Cossani, Gianfranco & Codoceo, Loreto & Cáceres, Hernán & Tabilo, Jorge, 2022. "Technical efficiency in Chile’s higher education system: A comparison of rankings and accreditation," Evaluation and Program Planning, Elsevier, vol. 92(C).
    8. Cui, Yuan & Ren, Xian-tong & He, Xi-jie & Yang, Guo-liang, 2023. "Is human and financial investment in Chinese universities effective?," Socio-Economic Planning Sciences, Elsevier, vol. 88(C).
    9. Zihong Liu & Haitao Xiong & Ying Sun, 2023. "Will Online MOOCs Improve the Efficiency of Chinese Higher Education Institutions? An Empirical Study Based on DEA," Sustainability, MDPI, vol. 15(7), pages 1-21, March.
    10. Wang, Derek D., 2019. "Performance-based resource allocation for higher education institutions in China," Socio-Economic Planning Sciences, Elsevier, vol. 65(C), pages 66-75.
    11. Fritz Schiltz & Kristof Witte & Deni Mazrekaj, 2020. "Managerial efficiency and efficiency differentials in adult education: a conditional and bias-corrected efficiency analysis," Annals of Operations Research, Springer, vol. 288(2), pages 529-546, May.
    12. Ruiz, José L. & Segura, José V. & Sirvent, Inmaculada, 2015. "Benchmarking and target setting with expert preferences: An application to the evaluation of educational performance of Spanish universities," European Journal of Operational Research, Elsevier, vol. 242(2), pages 594-605.
    13. Bornmann, Lutz & Gralka, Sabine & Anegón, Félix de Moya & Wohlrabe, Klaus, 2023. "Efficiency of universities and research-focused institutions worldwide: The introduction of a new input indicator reflecting institutional staff numbers," Journal of Informetrics, Elsevier, vol. 17(2).
    14. Drebee, Hayder Abbas & Abdul Razak, Nor Azam, 2018. "Measuring the Efficiency of Colleges at the University of Al-Qadisiyah-Iraq: A Data Envelopment Analysis Approach," Jurnal Ekonomi Malaysia, Faculty of Economics and Business, Universiti Kebangsaan Malaysia, vol. 52(3), pages 153-166.
    15. Mergoni, Anna & Soncin, Mara & Agasisti, Tommaso, 2023. "The effect of ICT on schools’ efficiency: Empirical evidence on 23 European countries," Omega, Elsevier, vol. 119(C).
    16. Alexander Arévalo S & Víctor Giménez G & Diego Prior J, 2022. "Análisis de eficiencia en educación: una aplicación del método StoNED," Revista Desarrollo y Sociedad, Universidad de los Andes,Facultad de Economía, CEDE, vol. 92(2), pages 45-91, October.
    17. Shamohammadi, Mehdi & Oh, Dong-hyun, 2019. "Measuring the efficiency changes of private universities of Korea: A two-stage network data envelopment analysis," Technological Forecasting and Social Change, Elsevier, vol. 148(C).
    18. Qiwei Xie & Yuanyuan Li & Lizheng Wang & Chao Liu, 2018. "Improving discrimination in data envelopment analysis without losing information based on Renyi’s entropy," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 26(4), pages 1053-1068, December.
    19. Marcel Clermont & Alexander Dirksen & Harald Dyckhoff, 2015. "Returns to scale of Business Administration research in Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 583-614, May.
    20. Sanjeet Singh & Prabhat Ranjan, 2018. "Efficiency analysis of non-homogeneous parallel sub-unit systems for the performance measurement of higher education," Annals of Operations Research, Springer, vol. 269(1), pages 641-666, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:8:d:10.1007_s11192-022-04436-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.