IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v47y2023i2p155-181.html
   My bibliography  Save this article

Evaluating Research Efficiency in Indonesian Higher Education Institution

Author

Listed:
  • Muhammad Dimyati
  • Adhi Indra Hermanu

Abstract

The Indonesian government has evaluated the research performance of universities, whose measurement process is projected into resources, management, outputs, and revenues to determine the provision of incentives, grants, and program funding to universities. However, efficiency calculations have shown that the outputs and competition-based incentives that drive scientific productivity are more complex. The most competitive systems must also be the most productive when considering resources. This study aimed to analyze the research efficiency in the Indonesian higher education system. The efficiency was analyzed by maximizing the 13 product outputs from the research budget and university staff. The result was then compared with the existing performance measurement analysis. Data Envelopment Analysis (DEA) was used to evaluate the efficiency based on the data of 47 universities in the Mandiri cluster and 144 in the Utama cluster for the 2014–2018 period. These findings showed that about 68% of universities have an efficiency value of 1 for the Mandiri group, almost 40% in the Utama group, and 41% for the two groups combined. Additionally, this study compared the efficiency analysis and the impact of the performance evaluation. The comparison showed that adding efficiency or productivity factors in the performance evaluation assessment produced a more accurate result.

Suggested Citation

  • Muhammad Dimyati & Adhi Indra Hermanu, 2023. "Evaluating Research Efficiency in Indonesian Higher Education Institution," Evaluation Review, , vol. 47(2), pages 155-181, April.
    • Handle: RePEc:sae:evarev:v:47:y:2023:i:2:p:155-181
    DOI: 10.1177/0193841X221118181
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X221118181
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X221118181?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. William W. Cooper & Lawrence M. Seiford & Joe Zhu (ed.), 2011. "Handbook on Data Envelopment Analysis," International Series in Operations Research and Management Science, Springer, number 978-1-4419-6151-8, April.
    2. Benedetto Lepori & Aldo Geuna & Antonietta Mira, 2019. "Scientific output scales with resources. A comparison of US and European universities," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-18, October.
    3. Worthington, Andrew C. & Lee, Boon L., 2008. "Efficiency, technology and productivity change in Australian universities, 1998-2003," Economics of Education Review, Elsevier, vol. 27(3), pages 285-298, June.
    4. Avkiran, Necmi K., 2001. "Investigating technical and scale efficiencies of Australian Universities through data envelopment analysis," Socio-Economic Planning Sciences, Elsevier, vol. 35(1), pages 57-80, March.
    5. Lutz Bornmann & Klaus Wohlrabe & Felix Moya Anegon, 2017. "Calculating the excellence shift: How efficiently do institutions produce highly cited papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1859-1864, September.
    6. Alexandros M. Theodoridis & Md. Mazharul Anwar, 2011. "A comparison of DEA and SFA methods: a case study of farm households in Bangladesh," Journal of Developing Areas, Tennessee State University, College of Business, vol. 45(1), pages 95-110, July-Dece.
    7. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    8. Koen Jonkers & Thomas Zacharewicz, 2016. "Research Performance Based Funding Systems: a Comparative Assessment," JRC Research Reports JRC101043, Joint Research Centre.
    9. Saba Hinrichs-Krapels & Jonathan Grant, 2016. "Exploring the effectiveness, efficiency and equity (3e’s) of research and research impact assessment," Palgrave Communications, Palgrave Macmillan, vol. 2(1), pages 1-9, December.
    10. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    11. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    12. Ashkan Ebadi & Andrea Schiffauerova, 2016. "How to boost scientific production? A statistical analysis of research funding and other influencing factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1093-1116, March.
    13. E Bayraktar & E Tatoglu & S Zaim, 2013. "Measuring the relative efficiency of quality management practices in Turkish public and private universities," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 64(12), pages 1810-1830, December.
    14. Korhonen, Pekka & Tainio, Risto & Wallenius, Jyrki, 2001. "Value efficiency analysis of academic research," European Journal of Operational Research, Elsevier, vol. 130(1), pages 121-132, April.
    15. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    16. Sandström, Ulf & Van den Besselaar, Peter, 2018. "Funding, evaluation, and the performance of national research systems," Journal of Informetrics, Elsevier, vol. 12(1), pages 365-384.
    17. Tommaso Agasisti & Geraint Johnes, 2009. "Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country," Education Economics, Taylor & Francis Journals, vol. 17(1), pages 59-79.
    18. Kulczycki, Emanuel & Korzeń, Marcin & Korytkowski, Przemysław, 2017. "Toward an excellence-based research funding system: Evidence from Poland," Journal of Informetrics, Elsevier, vol. 11(1), pages 282-298.
    19. Adhi Indra Hermanu & Mery Citra Sondari & Muhammad Dimyati & Diana Sari, 2022. "Study on university research performance based on systems theory: systematic literature review," International Journal of Productivity and Quality Management, Inderscience Enterprises Ltd, vol. 35(4), pages 447-472.
    20. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    21. Bolli, Thomas & Olivares, Maria & Bonaccorsi, Andrea & Daraio, Cinzia & Aracil, Adela Garcia & Lepori, Benedetto, 2016. "The differential effects of competitive funding on the production frontier and the efficiency of universities," Economics of Education Review, Elsevier, vol. 52(C), pages 91-104.
    22. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anis Mardiana Ahmad & Noor Saifurina Nana Khurizan, 2024. "Data Envelopment Analysis and Higher Education: A Systematic Review of the 2018–2022 Literature and Bibliometric Analysis of the Past 30 Years of Literature," SAGE Open, , vol. 14(3), pages 21582440241, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    2. Yang, Guo-liang & Fukuyama, Hirofumi & Song, Yao-yao, 2018. "Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model," Journal of Informetrics, Elsevier, vol. 12(1), pages 10-30.
    3. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    4. Ruiz, José L. & Segura, José V. & Sirvent, Inmaculada, 2015. "Benchmarking and target setting with expert preferences: An application to the evaluation of educational performance of Spanish universities," European Journal of Operational Research, Elsevier, vol. 242(2), pages 594-605.
    5. Sabine Gralka & Klaus Wohlrabe & Lutz Bornmann, 2018. "How to Measure Research Efficiency in Higher Education? Research Grants vs. Publication Output," CESifo Working Paper Series 7055, CESifo.
    6. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    7. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    8. Calogero Guccio & Marco Ferdinando Martorana & Isidoro Mazza, 2016. "Efficiency assessment and convergence in teaching and research in Italian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1063-1094, June.
    9. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    10. Mad Ithnin Salleh & Nurul Fadly Habidin & Abdul Halim Masnan & Nordin Mamat, 2017. "Estimating Technical Efficiency and Bootstrapping Malmquist Indices: Analysis of Malaysian Preschool Sector," International Journal of Academic Research in Business and Social Sciences, Human Resource Management Academic Research Society, International Journal of Academic Research in Business and Social Sciences, vol. 7(3), pages 440-457, March.
    11. Contreras, I. & Lozano, S., 2020. "Allocating additional resources to public universities. A DEA bargaining approach," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    12. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    13. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).
    14. Joanna Wolszczak-Derlacz & Aleksandra Parteka, 2011. "Efficiency of European public higher education institutions: a two-stage multicountry approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 887-917, December.
    15. Bornmann, Lutz & Gralka, Sabine & Anegón, Félix de Moya & Wohlrabe, Klaus, 2023. "Efficiency of universities and research-focused institutions worldwide: The introduction of a new input indicator reflecting institutional staff numbers," Journal of Informetrics, Elsevier, vol. 17(2).
    16. G. Thomas Sav, 2013. "Effects of Financial Source Dependency on Public University Operating Efficiencies: Data Envelopment Single-Stage and Tobit Two-Stage Evaluations," Review of Economics & Finance, Better Advances Press, Canada, vol. 3, pages 63-73, February.
    17. Kudła Janusz & Stachowiak-Kudła Monika & Figurski Adam, 2016. "Quality of Teaching and Research in Public Higher Education in Poland: Relationship with Financial Indicators and Efficiency," Journal of Management and Business Administration. Central Europe, Sciendo, vol. 24(4), pages 88-108, December.
    18. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    19. Cantner, Uwe & Grashof, Nils & Grebel, Thomas & Zhang, Xijie, 2023. "When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities," MPRA Paper 118139, University Library of Munich, Germany.
    20. Bergantino, Angela Stefania & Capozza, Claudia & Porcelli, Francesco, 2015. "Hotelling competition and teaching efficiency of Italian university faculties. A semi-parametric analysis," MPRA Paper 62927, University Library of Munich, Germany.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:47:y:2023:i:2:p:155-181. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.