IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_7055.html
   My bibliography  Save this paper

How to Measure Research Efficiency in Higher Education? Research Grants vs. Publication Output

Author

Listed:
  • Sabine Gralka
  • Klaus Wohlrabe
  • Lutz Bornmann

Abstract

An ongoing debate in the literature on efficiency of higher education institutions concerns the indicator for research output in the empirical analysis. While several studies chose to use the number of publications, others rely on the overall amount of research grants. The present study investigates whether both lead to similar or different assessments of universities. Besides the amount of research grants and the absolute number of publications, the number of publications belonging to the 10% and 1% most frequently cited papers in the corresponding subject category and publication year are evaluated. We show that there is a high correlation of efficiency values between the estimations using these indicators; however, the concordance is partly lower. The results do not only provide a helpful guideline for researchers, but are also valuable for policy makers deciding which incentives to create through funding.

Suggested Citation

  • Sabine Gralka & Klaus Wohlrabe & Lutz Bornmann, 2018. "How to Measure Research Efficiency in Higher Education? Research Grants vs. Publication Output," CESifo Working Paper Series 7055, CESifo.
    • Handle: RePEc:ces:ceswps:_7055
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp7055.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Johnes, Geraint & Johnes, Jill, 1993. "Measuring the Research Performance of UK Economics Departments: An Application of Data Envelopment Analysis," Oxford Economic Papers, Oxford University Press, vol. 45(2), pages 332-347, April.
    2. Andrew Worthington, 2001. "An Empirical Survey of Frontier Efficiency Measurement Techniques in Education," Education Economics, Taylor & Francis Journals, vol. 9(3), pages 245-268.
    3. Maria Olivares & Heike Wetzel, 2014. "Editor's Choice Competing in the Higher Education Market: Empirical Evidence for Economies of Scale and Scope in German Higher Education Institutions," CESifo Economic Studies, CESifo, vol. 60(4), pages 653-680.
    4. Tommaso Agasisti & Geraint Johnes, 2010. "Heterogeneity and the evaluation of efficiency: the case of Italian universities," Applied Economics, Taylor & Francis Journals, vol. 42(11), pages 1365-1375.
    5. Geraint Johnes & Astrid Schwarzenberger, 2011. "Differences in cost structure and the evaluation of efficiency: the case of German universities," Education Economics, Taylor & Francis Journals, vol. 19(5), pages 487-499, January.
    6. Tommaso Agasisti & Sabine Gralka, 2019. "The transient and persistent efficiency of Italian and German universities: a stochastic frontier analysis," Applied Economics, Taylor & Francis Journals, vol. 51(46), pages 5012-5030, October.
    7. Gerhard Kempkes & Carsten Pohl, 2010. "The efficiency of German universities-some evidence from nonparametric and parametric methods," Applied Economics, Taylor & Francis Journals, vol. 42(16), pages 2063-2079.
    8. Subal Kumbhakar & Gudbrand Lien & J. Hardaker, 2014. "Technical efficiency in competing panel data models: a study of Norwegian grain farming," Journal of Productivity Analysis, Springer, vol. 41(2), pages 321-337, April.
    9. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    10. Izadi, Hooshang & Johnes, Geraint & Oskrochi, Reza & Crouchley, Robert, 2002. "Stochastic frontier estimation of a CES cost function: the case of higher education in Britain," Economics of Education Review, Elsevier, vol. 21(1), pages 63-71, February.
    11. Johnes, Geraint, 1997. "Costs and Industrial Structure in Contemporary British Higher Education," Economic Journal, Royal Economic Society, vol. 107(442), pages 727-737, May.
    12. Worthington, Andrew C. & Lee, Boon L., 2008. "Efficiency, technology and productivity change in Australian universities, 1998-2003," Economics of Education Review, Elsevier, vol. 27(3), pages 285-298, June.
    13. Johnes, Jill, 2006. "Data envelopment analysis and its application to the measurement of efficiency in higher education," Economics of Education Review, Elsevier, vol. 25(3), pages 273-288, June.
    14. Alexander Eck & Sabine Gralka & Julia Heller, 2015. "Zur Effizienz der Hochschulen in den ost- und westdeutschen Flächenländern," ifo Dresden berichtet, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 22(02), pages 33-41, April.
    15. Maria Olivares & Heike Wetzel, 2011. "Competing in the Higher Education Market: Empirical Evidence for Economies of Scale and Scope in German Higher Education Institutions," Economics of Education Working Paper Series 0070, University of Zurich, Department of Business Administration (IBW).
    16. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    17. Klaus Wohlrabe & Lutz Bornmann & Sabine Gralka & Felix de Moya Anegon, 2018. "Wie effizient forschen Universitäten in Deutschland, deren Zukunftskonzepte im Rahmen der Exzellenzinitiative ausgezeichnet wurden?," ifo Working Paper Series 253, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    18. Susanne Warning, 2007. "The Economic Analysis of Universities," Books, Edward Elgar Publishing, number 4167.
    19. Gawellek, Bastian & Sunder, Marco, 2016. "The German excellence initiative and efficiency change among universities, 2001-2011," Working Papers 142, University of Leipzig, Faculty of Economics and Management Science.
    20. Gerhard Kempkes & Carsten Pohl, 2008. "Do Institutions Matter for University Cost Efficiency? Evidence from Germany," CESifo Economic Studies, CESifo, vol. 54(2), pages 177-203.
    21. Worthington, Andrew C. & Lee, Boon L., 2008. "Efficiency, technology and productivity change in Australian universities, 1998-2003," Economics of Education Review, Elsevier, vol. 27(3), pages 285-298, June.
    22. Gerhard Kempkes, 2008. "Do Institutions Matter for University Cost Efficiency? Evidence from Germany," CESifo Economic Studies, CESifo, vol. 54(2), pages 177-203, June.
    23. E Thanassoulis & M Kortelainen & G Johnes & J Johnes, 2011. "Costs and efficiency of higher education institutions in England: a DEA analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(7), pages 1282-1297, July.
    24. Bolli, Thomas & Olivares, Maria & Bonaccorsi, Andrea & Daraio, Cinzia & Aracil, Adela Garcia & Lepori, Benedetto, 2016. "The differential effects of competitive funding on the production frontier and the efficiency of universities," Economics of Education Review, Elsevier, vol. 52(C), pages 91-104.
    25. E Bayraktar & E Tatoglu & S Zaim, 2013. "Measuring the relative efficiency of quality management practices in Turkish public and private universities," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 64(12), pages 1810-1830, December.
    26. Sabine Gralka, 2018. "Persistent inefficiency in the higher education sector: evidence from Germany," Education Economics, Taylor & Francis Journals, vol. 26(4), pages 373-392, July.
    27. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    28. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    29. Tommaso Agasisti & Geraint Johnes, 2009. "Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country," Education Economics, Taylor & Francis Journals, vol. 17(1), pages 59-79.
    30. Fandel, Gunter, 2007. "On the performance of universities in North Rhine-Westphalia, Germany: Government's redistribution of funds judged using DEA efficiency measures," European Journal of Operational Research, Elsevier, vol. 176(1), pages 521-533, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Klaus Wohlrabe & Sabine Gralka, 2020. "Using archetypoid analysis to classify institutions and faculties of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 159-179, April.
    2. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).
    3. Eugénia de Matos Pedro & João Leitão & Helena Alves, 2021. "HEI Efficiency and Quality of Life: Seeding the Pro-Sustainability Efficiency," Sustainability, MDPI, vol. 13(2), pages 1-25, January.
    4. Manuel Salas-Velasco, 2020. "Measuring and explaining the production efficiency of Spanish universities using a non-parametric approach and a bootstrapped-truncated regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 825-846, February.
    5. Driss El Kadiri Boutchich, 2020. "Factors with significant impact on efficiency of research laboratories: case of the public university," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(4), pages 1317-1333, August.
    6. Klaus Wohlrabe & Félix de Moya Anegon & Lutz Bornmann, 2019. "How Efficiently Do Elite US Universities Produce Highly Cited Papers?," Publications, MDPI, vol. 7(1), pages 1-15, January.
    7. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    8. Klaus Wohlrabe & Sabine Gralka & Lutz Bornmann, 2019. "Zur Effizienz deutscher Universitäten und deren Entwicklung zwischen 2004 und 2015," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 72(21), pages 15-21, November.
    9. Sabine Gralka & Klaus Wohlrabe & Lutz Bornmann, 2018. "Benchmarking Research Performance in Efficiency Analyses: Third Party Funding vs Publications," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 71(16), pages 26-30, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gralka, Sabine, 2018. "Stochastic frontier analysis in higher education: A systematic review," CEPIE Working Papers 05/18, Technische Universität Dresden, Center of Public and International Economics (CEPIE).
    2. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.
    3. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    4. Wohlrabe, Klaus & Bornmann, Lutz & de Moya Anegon, Felix, 2017. "Wie effizient sind Universitäten in Deutschland, deren Zukunftskonzepte im Rahmen der Exzellenzinitiative ausgezeichnet wurden? Ein empirischer Vergleich von Input- und Output-Daten zur Forschung [," MPRA Paper 76218, University Library of Munich, Germany.
    5. Cantner, Uwe & Grashof, Nils & Grebel, Thomas & Zhang, Xijie, 2023. "When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities," MPRA Paper 118139, University Library of Munich, Germany.
    6. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    7. Klaus Wohlrabe & Lutz Bornmann & Sabine Gralka & Felix de Moya Anegon, 2018. "Wie effizient forschen Universitäten in Deutschland, deren Zukunftskonzepte im Rahmen der Exzellenzinitiative ausgezeichnet wurden?," ifo Working Paper Series 253, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    8. Tommaso Agasisti & Sabine Gralka, 2019. "The transient and persistent efficiency of Italian and German universities: a stochastic frontier analysis," Applied Economics, Taylor & Francis Journals, vol. 51(46), pages 5012-5030, October.
    9. Klaus Wohlrabe & Sabine Gralka & Lutz Bornmann, 2019. "Zur Effizienz deutscher Universitäten und deren Entwicklung zwischen 2004 und 2015," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 72(21), pages 15-21, November.
    10. Agasisti, Tommaso & Barra, Cristian & Zotti, Roberto, 2016. "Evaluating the efficiency of Italian public universities (2008–2011) in presence of (unobserved) heterogeneity," Socio-Economic Planning Sciences, Elsevier, vol. 55(C), pages 47-58.
    11. Erik E. Lehmann & Michele Meoli & Stefano Paleari & Sarah A. E. Stockinger, 2018. "Approaching effects of the economic crisis on university efficiency: a comparative study of Germany and Italy," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 8(1), pages 37-54, March.
    12. Vanesa D’Elia & Gustavo Ferro, 2019. "Empirical Efficiency Measurement in Higher Education: An Overview," CEMA Working Papers: Serie Documentos de Trabajo. 708, Universidad del CEMA.
    13. Sabine Gralka, 2018. "Persistent inefficiency in the higher education sector: evidence from Germany," Education Economics, Taylor & Francis Journals, vol. 26(4), pages 373-392, July.
    14. Joanna Wolszczak-Derlacz & Aleksandra Parteka, 2011. "Efficiency of European public higher education institutions: a two-stage multicountry approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(3), pages 887-917, December.
    15. Lutz Bornmann & Klaus Wohlrabe & Sabine Gralka, 2019. "The graduation shift of German universities of applied sciences," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-19, January.
    16. Barra, Cristian & Lagravinese, Raffaele & Zotti, Roberto, 2018. "Does econometric methodology matter to rank universities? An analysis of Italian higher education system," Socio-Economic Planning Sciences, Elsevier, vol. 62(C), pages 104-120.
    17. Tzeremes, Nickolaos & Halkos, George, 2010. "A DEA approach for measuring university departments’ efficiency," MPRA Paper 24029, University Library of Munich, Germany.
    18. Laureti, Tiziana & Secondi, Luca & Biggeri, Luigi, 2014. "Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach," Economics of Education Review, Elsevier, vol. 42(C), pages 147-164.
    19. Maria Olivares & Andrea Schenker-Wicki, 2012. "The Dynamics of Productivity in the Swiss and German University Sector: A Non-Parametric Analysis that Accounts for Heterogeneous Production," Working Papers 309, University of Zurich, Department of Business Administration (IBW).
    20. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2012. "Measuring Public Owned University Departments' Efficiency: A Bootstrapped DEA Approach," Journal of Economics and Econometrics, Economics and Econometrics Society, vol. 55(2), pages 1-24.

    More about this item

    Keywords

    efficiency; data envelopment analysis; stochastic frontier analysis; higher education; universities; Germany;
    All these keywords.

    JEL classification:

    • A23 - General Economics and Teaching - - Economic Education and Teaching of Economics - - - Graduate
    • H52 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Education
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • D61 - Microeconomics - - Welfare Economics - - - Allocative Efficiency; Cost-Benefit Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_7055. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.