IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v49y2020i8s004873332030161x.html
   My bibliography  Save this article

Higher education policy: Why hope for quality when rewarding quantity?

Author

Listed:
  • Civera, Alice
  • Lehmann, Erik E.
  • Paleari, Stefano
  • Stockinger, Sarah A.E.

Abstract

This paper analyses the impact of the Excellence Initiative (EI) in Germany, a policy intervention aimed to promote and select outstanding active research universities by competitively allocating additional public funds. Academic debate on efficiency and effectiveness of higher education policy does not addresses issues such as treatment and selection effects, suffers from generalizable measurement problems, and does not take a comparative approach. Our objective is to fill this gap by adopting Italy as a control country. In doing so, this paper examines (1) whether this policy approach is suitable to stimulate the system and the awarded institutions, (2) how the performance impact can be measured and (3) whether the results are driven by country specific effects or are generalizable. To this end, we applied a triple difference-in-differences analysis (DDD) on a dataset of 72 German and 51 Italian state universities during the first round of the EI, from 2004 to 2013. We found that the EI has had a positive effect on research quantity, but a reverse effect on research quality.

Suggested Citation

  • Civera, Alice & Lehmann, Erik E. & Paleari, Stefano & Stockinger, Sarah A.E., 2020. "Higher education policy: Why hope for quality when rewarding quantity?," Research Policy, Elsevier, vol. 49(8).
    • Handle: RePEc:eee:respol:v:49:y:2020:i:8:s004873332030161x
    DOI: 10.1016/j.respol.2020.104083
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S004873332030161X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2020.104083?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. David Audretsch & Erik Lehmann & Stefano Paleari, 2015. "Academic policy and entrepreneurship: a European perspective," The Journal of Technology Transfer, Springer, vol. 40(3), pages 363-368, June.
    2. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    3. Antreas Athanassopoulos & Estelle Shale, 1997. "Assessing the Comparative Efficiency of Higher Education Institutions in the UK by the Means of Data Envelopment Analysis," Education Economics, Taylor & Francis Journals, vol. 5(2), pages 117-134.
    4. Riphahn Regina T. & Thalmaier Anja, 2001. "Behavioral Effects of Probation Periods: An Analysis of Worker Absenteeism / Anreizeffekte der Probezeit: Eine Untersuchung von Fehlzeiten bei Arbeitnehmern," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 221(2), pages 179-201, April.
    5. Aldo Geuna, 2001. "The Changing Rationale for European University Research Funding: Are There Negative Unintended Consequences?," Journal of Economic Issues, Taylor & Francis Journals, vol. 35(3), pages 607-632, September.
    6. Aksnes, Dag W. & Rip, Arie, 2009. "Researchers' perceptions of citations," Research Policy, Elsevier, vol. 38(6), pages 895-905, July.
    7. Tommaso Agasisti & Carsten Pohl, 2012. "Comparing German and Italian Public Universities: Convergence or Divergence in the Higher Education Landscape?," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 33, pages 71-85, March.
    8. Erik E. Lehmann & Michele Meoli & Stefano Paleari & Sarah A. E. Stockinger, 2018. "Approaching effects of the economic crisis on university efficiency: a comparative study of Germany and Italy," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 8(1), pages 37-54, March.
    9. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    10. Matthias Menter & Erik E. Lehmann & Torben Klarl, 2018. "In search of excellence: a case study of the first excellence initiative of Germany," Journal of Business Economics, Springer, vol. 88(9), pages 1105-1132, December.
    11. Vogel, Rick & Hattke, Fabian & Petersen, Jessica, 2017. "Journal rankings in management and business studies: What rules do we play by?," Research Policy, Elsevier, vol. 46(10), pages 1707-1722.
    12. J Sylvan Katz, 2000. "Scale-independent indicators and research evaluation," Science and Public Policy, Oxford University Press, vol. 27(1), pages 23-36, February.
    13. David H. Autor, 2003. "Outsourcing at Will: The Contribution of Unjust Dismissal Doctrine to the Growth of Employment Outsourcing," Journal of Labor Economics, University of Chicago Press, vol. 21(1), pages 1-42, January.
    14. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2011. "Multivariate Matching Methods That Are Monotonic Imbalance Bounding," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 345-361.
    15. Kerstin Bruckmeier & Georg-Benedikt Fischer & Berthold U. Wigger, 2017. "Status effects of the German Excellence Initiative," International Journal of Economics and Finance, Canadian Center of Science and Education, vol. 9(3), pages 177-183, March.
    16. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    17. Annika C. Froehlich, 2016. "Does the curricular structure affect doctoral enrolment?," Journal of Business Economics, Springer, vol. 86(9), pages 1067-1089, December.
    18. Nabil Amara & Réjean Landry & Norrin Halilem, 2015. "What can university administrators do to increase the publication and citation scores of their faculty members?," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 489-530, May.
    19. Geuna, Aldo & Nesta, Lionel J.J., 2006. "University patenting and its effects on academic research: The emerging European evidence," Research Policy, Elsevier, vol. 35(6), pages 790-807, July.
    20. Liping, Yu & Yuqing, Chen & Yuntao, Pan & Yishan, Wu, 2009. "Research on the evaluation of academic journals based on structural equation modeling," Journal of Informetrics, Elsevier, vol. 3(4), pages 304-311.
    21. Autio, Erkko & Rannikko, Heikki, 2016. "Retaining winners: Can policy boost high-growth entrepreneurship?," Research Policy, Elsevier, vol. 45(1), pages 42-55.
    22. Gawellek, Bastian & Sunder, Marco, 2016. "The German excellence initiative and efficiency change among universities, 2001-2011," Working Papers 142, University of Leipzig, Faculty of Economics and Management Science.
    23. He, Zi-Lin & Geng, Xue-Song & Campbell-Hunt, Colin, 2009. "Research collaboration and research output: A longitudinal study of 65 biomedical scientists in a New Zealand university," Research Policy, Elsevier, vol. 38(2), pages 306-317, March.
    24. Zoltan J. Acs & Luc Anselin & Attila Varga, 2008. "Patents and Innovation Counts as Measures of Regional Production of New Knowledge," Chapters, in: Entrepreneurship, Growth and Public Policy, chapter 11, pages 135-151, Edward Elgar Publishing.
    25. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    26. Richard Harrison & Claire Leitch, 2010. "Voodoo Institution or Entrepreneurial University? Spin-off Companies, the Entrepreneurial System and Regional Development in the UK," Regional Studies, Taylor & Francis Journals, vol. 44(9), pages 1241-1262.
    27. Agrawal, Ajay & McHale, John & Oettl, Alexander, 2017. "How stars matter: Recruiting and peer effects in evolutionary biology," Research Policy, Elsevier, vol. 46(4), pages 853-867.
    28. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    29. Etzkowitz, Henry & Leydesdorff, Loet, 2000. "The dynamics of innovation: from National Systems and "Mode 2" to a Triple Helix of university-industry-government relations," Research Policy, Elsevier, vol. 29(2), pages 109-123, February.
    30. Jennifer Brown, 2011. "Quitters Never Win: The (Adverse) Incentive Effects of Competing with Superstars," Journal of Political Economy, University of Chicago Press, vol. 119(5), pages 982-1013.
    31. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    32. Gerhard Kempkes & Carsten Pohl, 2010. "The efficiency of German universities-some evidence from nonparametric and parametric methods," Applied Economics, Taylor & Francis Journals, vol. 42(16), pages 2063-2079.
    33. Ben R. Martin, 2012. "Are universities and university research under threat? Towards an evolutionary model of university speciation," Cambridge Journal of Economics, Oxford University Press, vol. 36(3), pages 543-565.
    34. Li, Eldon Y. & Liao, Chien Hsiang & Yen, Hsiuju Rebecca, 2013. "Co-authorship networks and research impact: A social capital perspective," Research Policy, Elsevier, vol. 42(9), pages 1515-1530.
    35. Geuna, Aldo & Piolatto, Matteo, 2016. "Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)," Research Policy, Elsevier, vol. 45(1), pages 260-271.
    36. Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    37. Bolli, Thomas & Somogyi, Frank, 2011. "Do competitively acquired funds induce universities to increase productivity?," Research Policy, Elsevier, vol. 40(1), pages 136-147, February.
    38. De Filippo, D. & Casani, F. & Sanz-Casado, E., 2016. "University excellence initiatives in Spain, a possible strategy for optimising resources and improving local performance," Technological Forecasting and Social Change, Elsevier, vol. 113(PB), pages 185-194.
    39. Jauernig, Johanna & Uhl, Matthias & Luetge, Christoph, 2016. "Competition-induced punishment of winners and losers: Who is the target?," Journal of Economic Psychology, Elsevier, vol. 57(C), pages 13-25.
    40. Dietmar Braun & David H Guston, 2003. "Principal-agent theory and research policy: An introduction," Science and Public Policy, Oxford University Press, vol. 30(5), pages 302-308, October.
    41. Tommaso Agasisti & Giuseppe Catalano & Paolo Landoni & Roberto Verganti, 2012. "Evaluating the performance of academic departments: an analysis of research-related output efficiency," Research Evaluation, Oxford University Press, vol. 21(1), pages 2-14, February.
    42. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    43. Torger Möller & Marion Schmidt & Stefan Hornbostel, 2016. "Assessing the effects of the German Excellence Initiative with bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2217-2239, December.
    44. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    45. Charnes, A. & Cooper, W. W. & Rhodes, E., 1979. "Measuring the efficiency of decision-making units," European Journal of Operational Research, Elsevier, vol. 3(4), pages 339-338, July.
    46. Debackere, Koenraad & Rappa, Michael A., 1994. "Institutional variations in problem choice and persistence among scientists in an emerging field," Research Policy, Elsevier, vol. 23(4), pages 425-441, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. James A. Cunningham & Erik E. Lehmann & Matthias Menter, 2022. "The organizational architecture of entrepreneurial universities across the stages of entrepreneurship: a conceptual framework," Small Business Economics, Springer, vol. 59(1), pages 11-27, June.
    2. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    3. Hanna Hottenrott & Michael E. Rose & Cornelia Lawson, 2021. "The rise of multiple institutional affiliations in academia," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(8), pages 1039-1058, August.
    4. van Dalen, Hendrik Peter, 2021. "How the publish-or-perish principle divides a science: The case of economists," Other publications TiSEM a6a5a855-bb5a-4d52-a841-3, Tilburg University, School of Economics and Management.
    5. Stepan Kavan, 2021. "Evaluation of the Current Approach to Education of Security Issues at Selected Universities Preparing Future Pedagogues," Sustainability, MDPI, vol. 13(19), pages 1-12, September.
    6. Ryazanova, Olga & Jaskiene, Jolanta, 2022. "Managing individual research productivity in academic organizations: A review of the evidence and a path forward," Research Policy, Elsevier, vol. 51(2).
    7. Mario Fernandes & Andreas Walter, 2023. "The times they are a-changin’: profiling newly tenured business economics professors in Germany over the past thirty years," Journal of Business Economics, Springer, vol. 93(5), pages 929-971, July.
    8. Püttmann, Vitus & Thomsen, Stephan L. & Trunzer, Johannes, 2020. "Zur Relevanz von Ausstattungsunterschieden für Forschungsleistungsvergleiche: Ein Diskussionsbeitrag für die Wirtschaftswissenschaften in Deutschland," Hannover Economic Papers (HEP) dp-679, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Mar 2021.
    9. Aslıhan Sezgin & Keziban Orbay & Metin Orbay, 2022. "Educational Research Review From Diverse Perspectives: A Bibliometric Analysis of Web of Science (2011–2020)," SAGE Open, , vol. 12(4), pages 21582440221, December.
    10. Matveeva, Nataliya & Sterligov, Ivan & Yudkevich, Maria, 2021. "The effect of Russian University Excellence Initiative on publications and collaboration patterns," Journal of Informetrics, Elsevier, vol. 15(1).
    11. Seokbeom Kwon & Kazuyuki Motohashi & Kenta Ikeuchi, 2022. "Chasing two hares at once? Effect of joint institutional change for promoting commercial use of university knowledge and scientific research," The Journal of Technology Transfer, Springer, vol. 47(4), pages 1242-1272, August.
    12. Marton Demeter & Agnes Jele & Zsolt Balázs Major, 2022. "The model of maximum productivity for research universities SciVal author ranks, productivity, university rankings, and their implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4335-4361, August.
    13. Civera, Alice & Lehmann, Erik E & Meoli, Michele & Paleari, Stefano Paleari, 2023. "The Attractiveness of European Higher Education Systems: A Comparative Analysis of Faculty Remuneration and Career Paths," University of California at Berkeley, Center for Studies in Higher Education qt08x00432, Center for Studies in Higher Education, UC Berkeley.
    14. Hanna Hottenrott & Cornelia Lawson, 2022. "What is behind multiple institutional affiliations in academia?," Science and Public Policy, Oxford University Press, vol. 49(3), pages 382-402.
    15. Natalya Radko & Maksim Belitski & Yelena Kalyuzhnova, 2023. "Conceptualising the entrepreneurial university: the stakeholder approach," The Journal of Technology Transfer, Springer, vol. 48(3), pages 955-1044, June.
    16. Hendrik P. Dalen, 2021. "How the publish-or-perish principle divides a science: the case of economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1675-1694, February.
    17. Mehmet Pinar, 2023. "Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5731-5766, October.
    18. Aloys Prinz & Thomas Ehrmann, 2022. "Academia as a league system," Journal of Business Economics, Springer, vol. 92(7), pages 1065-1092, September.
    19. Rene Brauer & Mirek Dymitrow, 2020. "The Language of Sustainable Tourism as a Proxy Indicator of Quality," Sustainability, MDPI, vol. 13(1), pages 1-17, December.
    20. Shengbo Liu & Xiaoting Luo & Miaomiao Liu, 2023. "Was Chinese “Double-First Class” Construction Policy Influential? Analysis Using Propensity Score Matching," Sustainability, MDPI, vol. 15(8), pages 1-13, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Erik E. Lehmann & Michele Meoli & Stefano Paleari & Sarah A. E. Stockinger, 2018. "Approaching effects of the economic crisis on university efficiency: a comparative study of Germany and Italy," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 8(1), pages 37-54, March.
    2. Cantner, Uwe & Grashof, Nils & Grebel, Thomas & Zhang, Xijie, 2023. "When Excellence is not Excellent: The Impact of the Excellence Initiative on the Relative Productivity of German Universities," MPRA Paper 118139, University Library of Munich, Germany.
    3. Mehdi Rhaiem, 2017. "Measurement and determinants of academic research efficiency: a systematic review of the evidence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 581-615, February.
    4. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    5. Calogero Guccio & Marco Ferdinando Martorana & Isidoro Mazza, 2016. "Efficiency assessment and convergence in teaching and research in Italian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1063-1094, June.
    6. Amara, Nabil & Rhaiem, Mehdi & Halilem, Norrin, 2020. "Assessing the research efficiency of Canadian scholars in the management field: Evidence from the DEA and fsQCA," Journal of Business Research, Elsevier, vol. 115(C), pages 296-306.
    7. Bergantino, Angela Stefania & Capozza, Claudia & Porcelli, Francesco, 2015. "Hotelling competition and teaching efficiency of Italian university faculties. A semi-parametric analysis," MPRA Paper 62927, University Library of Munich, Germany.
    8. Lars Herberholz & Berthold U. Wigger, 2020. "Efficiency of European Universities: A Comparison of Peers," CESifo Working Paper Series 8044, CESifo.
    9. Matthias Menter & Erik E. Lehmann & Torben Klarl, 2018. "In search of excellence: a case study of the first excellence initiative of Germany," Journal of Business Economics, Springer, vol. 88(9), pages 1105-1132, December.
    10. Calogero Guccio & Marco Ferdinando Martorana & Luisa Monaco, 2016. "Evaluating the impact of the Bologna Process on the efficiency convergence of Italian universities: a non-parametric frontier approach," Journal of Productivity Analysis, Springer, vol. 45(3), pages 275-298, June.
    11. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    12. Herberholz, Lars & Wigger, Berthold U., 2021. "Efficiency of European universities: A comparison of peers," Research Policy, Elsevier, vol. 50(9).
    13. Mario Fernandes & Andreas Walter, 2023. "The times they are a-changin’: profiling newly tenured business economics professors in Germany over the past thirty years," Journal of Business Economics, Springer, vol. 93(5), pages 929-971, July.
    14. Benedetto Lepori & Aldo Geuna & Valerio Veglio, 2017. "A Typology of European Research Universities. Differentiation, Layering and Resource Distribution," SPRU Working Paper Series 2017-01, SPRU - Science Policy Research Unit, University of Sussex Business School.
    15. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    16. Чернова Е. Г. & Ахобадзе Т. Д. & Малова А. С. & Салтан А. А., 2017. "Модели Финансирования Высшего Образования И Эффективность Деятельности Университетов Эмпирическое Исследование Европейского Опыта И Отечественная Практика," Вопросы образования // Educational Studies Moscow, National Research University Higher School of Economics, issue 3, pages 37-82.
    17. Klaus Wohlrabe & Sabine Gralka & Lutz Bornmann, 2019. "Zur Effizienz deutscher Universitäten und deren Entwicklung zwischen 2004 und 2015," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 72(21), pages 15-21, November.
    18. Boutheina Bannour & Asma Sghaier & Mohammad Nurunnabi, 2020. "How to Choose a Nonparametric Frontier Model? Technical Efficiency of Turkish Banks Assessing Global," Global Business Review, International Management Institute, vol. 21(2), pages 348-364, April.
    19. Benedetto, Lepori & Geuna, Aldo & Veglio, Valerio, 2017. "A Typology of European Universities. Differentiation and resource distribution," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201705, University of Turin.
    20. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.

    More about this item

    Keywords

    Higher education; Excellence initiative; Policy intervention; Research quantity; Research quality;
    All these keywords.

    JEL classification:

    • H52 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:49:y:2020:i:8:s004873332030161x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.