IDEAS home Printed from https://ideas.repec.org/p/vuw/vuwcpf/20257.html
   My bibliography  Save this paper

The Evolution of Research Quality in New Zealand Universities as Measured by the Performance-Based Research Fund Process

Author

Listed:
  • Buckle, Robert A
  • Creedy, John

Abstract

This paper examines how the research quality of staff within New Zealand universities has evolved since the introduction in 2003 of the Performance-based Research Fund (PBRF). The analysis uses a database consisting of an anonymous ‘quality evaluation category (QEC) for each individual assessed in each of the three PBRF assessment rounds. Emphasis is on the evaluation of organisational changes in performance. The paper examines the extent to which each university’s Average Quality Score (AQS) changed as a result of changes in the QECs of existing staff over time and from the exit and entry of staff with different scores. The sensitivity of university rankings to the cardinal scale used by the PBRF is also considered and the degree of convergence amongst the universities is assessed. The data also include information about the age of staff evaluated in PBRF, and this is used to evaluate changes in the age distribution of staff across universities, and the ages of those making transitions within universities and between grades. The results reveal a systematic ageing of university staff in NZ and a significant change in the grade distribution by age, and age distribution by grade. A number of hypotheses regarding organisation change in response to the introduction of PBRF are discussed and tested by comparing universities with different patterns of change.

Suggested Citation

  • Buckle, Robert A & Creedy, John, 2017. "The Evolution of Research Quality in New Zealand Universities as Measured by the Performance-Based Research Fund Process," Working Paper Series 20257, Victoria University of Wellington, Chair in Public Finance.
    • Handle: RePEc:vuw:vuwcpf:20257
    as

    Download full text from publisher

    File URL: https://ir.wgtn.ac.nz/handle/123456789/20257
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Abramovitz, Moses, 1986. "Catching Up, Forging Ahead, and Falling Behind," The Journal of Economic History, Cambridge University Press, vol. 46(2), pages 385-406, June.
    2. David L. Anderson & Warren Smart & John Tressler, 2013. "Evaluating research -- peer review team assessment and journal based bibliographic measures: New Zealand PBRF research output scores in 2006," New Zealand Economic Papers, Taylor & Francis Journals, vol. 47(2), pages 140-157, August.
    3. Chavarro, Diego & Tang, Puay & Ràfols, Ismael, 2017. "Why researchers publish in non-mainstream journals: Training, knowledge bridging, and gap filling," Research Policy, Elsevier, vol. 46(9), pages 1666-1680.
    4. Tim Hazledine & Cliff Kurniawan, 2005. "Efficiency of New Zealand universities and the impact of the performance based research fund," New Zealand Economic Papers, Taylor & Francis Journals, vol. 39(2), pages 153-179.
    5. A. Peter W. Hodder & Catherine Hodder, 2010. "Research culture and New Zealand’s performance-based research fund: some insights from bibliographic compilations of research outputs," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 887-901, September.
    6. John Gibson & John Tressler & David L. Anderson, 2008. "Do Research Assessment Exercises Raise the Returns to Publication Quality? Evidence from the New Zealand Market for Academic Economists," Working Papers in Economics 08/11, University of Waikato.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.
    2. Robert A. Buckle & John Creedy, 2020. "The ‘disciplinary effect’ of the performance-based research fund process in New Zealand," New Zealand Economic Papers, Taylor & Francis Journals, vol. 54(2), pages 107-126, May.
    3. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    4. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    5. Robert A. Buckle & John Creedy & Norman Gemmell, 2020. "Is external research assessment associated with convergence or divergence of research quality across universities and disciplines? Evidence from the PBRF process in New Zealand," Applied Economics, Taylor & Francis Journals, vol. 52(36), pages 3919-3932, July.
    6. Robert A. Buckle and John Creedy, 2018. "The Impact on Research Quality of Performance-Based Funding: The Case of New Zealand’s PBRF Scheme," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 24(1), pages 25-48.
    7. Buckle, Robert A & Creedy, John, 2022. "The Performance Based Research Fund in NZ: Taking Stock and Looking Forward," Working Paper Series 21354, Victoria University of Wellington, Chair in Public Finance.
    8. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    9. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert A. Buckle & John Creedy, 2019. "The evolution of research quality in New Zealand universities as measured by the performance-based research fund process," New Zealand Economic Papers, Taylor & Francis Journals, vol. 53(2), pages 144-165, May.
    2. David L. Anderson & John Tressler, 2013. "The New Zealand performance-based research fund and its impact on publication activity in economics," Research Evaluation, Oxford University Press, vol. 23(1), pages 1-11, September.
    3. Robert A. Buckle and John Creedy, 2018. "The Impact on Research Quality of Performance-Based Funding: The Case of New Zealand’s PBRF Scheme," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 24(1), pages 25-48.
    4. David L. Anderson & John Tressler, 2017. "Researcher rank stability across alternative output measurement schemes in the context of a time limited research evaluation: the New Zealand case," Applied Economics, Taylor & Francis Journals, vol. 49(45), pages 4542-4553, September.
    5. Robert A. Buckle and John Creedy, 2018. "The Impact on Research Quality of Performance-Based Funding: The Case of New Zealand’s PBRF Scheme," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 24(1), pages 25-48.
    6. David L. Anderson & John Tressler, 2015. "Are Researcher Rankings Stable Across Alternative Output Measurement Schemes in the Context of a Time Limited Research Evaluation? The New Zealand Case," Working Papers in Economics 15/10, University of Waikato.
    7. repec:ilo:ilowps:366690 is not listed on IDEAS
    8. Dowling, Malcolm & Ray, David, 2000. "The structure and composition of international trade in Asia:: historical trends and future prospects," Journal of Asian Economics, Elsevier, vol. 11(3), pages 301-318, December.
    9. Bryce, Cormac & Dowling, Michael & Lucey, Brian, 2020. "The journal quality perception gap," Research Policy, Elsevier, vol. 49(5).
    10. Jan Fagerberg & Martin Srholec, 2017. "Global Dynamics, Capabilities and the Crisis," Economic Complexity and Evolution, in: Andreas Pyka & Uwe Cantner (ed.), Foundations of Economic Change, pages 83-106, Springer.
    11. Flora Bellone, 1997. "Les difficultés de la réalisation du rattrapage technologique. Une approche théorique de la convergence conditionnelle," Revue Économique, Programme National Persée, vol. 48(3), pages 409-418.
    12. Andreja Benkovic & Juan Felipe Mejía, 2008. "Tourism as a driver of economic development: The Colombian experience," Documentos de Trabajo CIEF 10630, Universidad EAFIT.
    13. Marattin, Luigi & Marzo, Massimiliano & Zagaglia, Paolo, 2013. "Distortionary tax instruments and implementable monetary policy," International Review of Economics & Finance, Elsevier, vol. 25(C), pages 219-243.
    14. Can Huang & Naubahar Sharif, 2016. "Global technology leadership: The case of China," Science and Public Policy, Oxford University Press, vol. 43(1), pages 62-73.
    15. Alessandro STERLACCHINI, 2006. "Innovation, Knowledge and Regional Economic Performances: Regularities and Differences in the EU," Working Papers 260, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    16. Giuseppina Guagnano & Elisabetta Santarelli & Isabella Santini, 2016. "Can Social Capital Affect Subjective Poverty in Europe? An Empirical Analysis Based on a Generalized Ordered Logit Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 128(2), pages 881-907, September.
    17. Jens J. Krüger, 2020. "Long‐run productivity trends: A global update with a global index," Review of Development Economics, Wiley Blackwell, vol. 24(4), pages 1393-1412, November.
    18. Prados de la Escosura, Leandro & Rosés, Joan R., 2008. "Proximate causes of economic growth in Spain, 1850-2000," IFCS - Working Papers in Economic History.WH wp08-12, Universidad Carlos III de Madrid. Instituto Figuerola.
    19. Andrés Rodríguez-Pose & Fabrice Comptour, 2010. "Do clusters generate greater innovation and growth? An analysis of European regions," Working Papers 2010-15, Instituto Madrileño de Estudios Avanzados (IMDEA) Ciencias Sociales.
    20. Christoph Meister & Bart Verspagen & Guntram B. Wolff, 2006. "European Productivity Gaps: Is R&D the Solution?," Chapters, in: Susanne Mundschenk & Michael H. Stierle & Ulrike Stierle-von Schütz & Iulia Traistaru-Siedschlag (ed.), Competitiveness and Growth in Europe, chapter 8, Edward Elgar Publishing.
    21. Davide Fiaschi & Andrea Mario Lavezzi & Angela Parenti, 2020. "Deep and Proximate Determinants of the World Income Distribution," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 66(3), pages 677-710, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vuw:vuwcpf:20257. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Library Technology Services (email available below). General contact details of provider: https://edirc.repec.org/data/fcvuwnz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.