IDEAS home Printed from https://ideas.repec.org/p/vuw/vuwcpf/20931.html
   My bibliography  Save this paper

Is External Research Assessment Associated with Convergence or Divergence of Research Quality Across Universities and Disciplines? Evidence from the PBRF Process in New Zealand

Author

Listed:
  • Buckle, Robert A.
  • Creedy, John
  • Gemmell, Norman

Abstract

Performance-based research quality measures have been adopted in many countries as a basis for allocating funding to universities. The question arises of whether this produces a divergence of research quality across universities and academic disciplines, or convergence whereby initially lower-quality institutions and disciplines catch-up? This paper examines whether the introduction of the New Zealand Performance-Based Research Fund process produced convergence or divergence in research quality scores of universities and disciplines between the 2003 and 2012 assessments. Anonymous individual researcher quality scores in 2003 and 2012 were used to derive average quality scores for disciplines and universities. Substantial convergence in average research quality is found over the period. With few exceptions, the hypothesis that rates of convergence have been uniform across almost all universities and disciplines is supported.

Suggested Citation

  • Buckle, Robert A. & Creedy, John & Gemmell, Norman, 2019. "Is External Research Assessment Associated with Convergence or Divergence of Research Quality Across Universities and Disciplines? Evidence from the PBRF Process in New Zealand," Working Paper Series 20931, Victoria University of Wellington, Chair in Public Finance.
  • Handle: RePEc:vuw:vuwcpf:20931
    as

    Download full text from publisher

    File URL: https://ir.wgtn.ac.nz/handle/123456789/20931
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Robert A. Buckle and John Creedy, 2018. "The Impact on Research Quality of Performance-Based Funding: The Case of New Zealand’s PBRF Scheme," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 24(1), pages 25-48.
    2. repec:clg:wpaper:2009-04 is not listed on IDEAS
    3. Robert A. Buckle & John Creedy, 2020. "The ‘disciplinary effect’ of the performance-based research fund process in New Zealand," New Zealand Economic Papers, Taylor & Francis Journals, vol. 54(2), pages 107-126, May.
    4. Abramovitz, Moses, 1986. "Catching Up, Forging Ahead, and Falling Behind," The Journal of Economic History, Cambridge University Press, vol. 46(2), pages 385-406, June.
    5. Dowrick, Steve & Gemmell, Norman, 1991. "Industrialisation, Catching Up and Economic Growth: A Comparative Study across the World's Capitalist Economies," Economic Journal, Royal Economic Society, vol. 101(405), pages 263-275, March.
    6. Abbott, M. & Doucouliagos, C., 2003. "The efficiency of Australian universities: a data envelopment analysis," Economics of Education Review, Elsevier, vol. 22(1), pages 89-97, February.
    7. Castle Jennifer L. & Doornik Jurgen A & Hendry David F., 2011. "Evaluating Automatic Model Selection," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-33, February.
    8. A. Abigail Payne & Joanne Roberts, 2010. "Government Oversight of Public Universities: Are Centralized Performance Schemes Related to Increased Quantity or Quality?," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 207-212, February.
    9. Quah, D., 1990. "Galton'S Fallacy And The Tests Of The Convergence Hypothesis," Working papers 552, Massachusetts Institute of Technology (MIT), Department of Economics.
    10. Robert A. Buckle & John Creedy, 2019. "The evolution of research quality in New Zealand universities as measured by the performance-based research fund process," New Zealand Economic Papers, Taylor & Francis Journals, vol. 53(2), pages 144-165, May.
    11. Robert A. Buckle and John Creedy, 2018. "The Impact on Research Quality of Performance-Based Funding: The Case of New Zealand’s PBRF Scheme," Agenda - A Journal of Policy Analysis and Reform, Australian National University, College of Business and Economics, School of Economics, vol. 24(1), pages 25-48.
    12. Jung Cheol Shin & William K. Cummings, 2010. "Multilevel analysis of academic publishing across disciplines: research preference, collaboration, and time on research," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 581-594, November.
    13. Robert A. Buckle & John Creedy, 2019. "An evaluation of metrics used by the Performance-based Research Fund process in New Zealand," New Zealand Economic Papers, Taylor & Francis Journals, vol. 53(3), pages 270-287, September.
    14. Malcolm Abbott & Chris Doucouliagos, 2000. "Technical and scale efficiency of vocational education and training institutions: The case of the New Zealand polytechnics," New Zealand Economic Papers, Taylor & Francis Journals, vol. 34(1), pages 1-23.
    15. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    16. Boyle Glenn, 2008. "Pay Peanuts and Get Monkeys? Evidence from Academia," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(1), pages 1-26, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.
    2. Mehmet Pinar, 2023. "Do research performances of universities and disciplines in England converge or diverge? An assessment of the progress between research excellence frameworks in 2014 and 2021," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5731-5766, October.
    3. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    4. Buckle, Robert A & Creedy, John, 2022. "The Performance Based Research Fund in NZ: Taking Stock and Looking Forward," Working Paper Series 21354, Victoria University of Wellington, Chair in Public Finance.
    5. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    6. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    7. Daniele Checchi & Irene Mazzotta & Sandro Momigliano & Francesco Olivanti, 2020. "Convergence or polarisation? The impact of research assessment exercises in the Italian case," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1439-1455, August.
    8. Robert A. Buckle & John Creedy, 2022. "Methods to evaluate institutional responses to performance‐based research funding systems," Australian Economic Papers, Wiley Blackwell, vol. 61(3), pages 615-634, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert A. Buckle & John Creedy & Norman Gemmell, 2020. "Is external research assessment associated with convergence or divergence of research quality across universities and disciplines? Evidence from the PBRF process in New Zealand," Applied Economics, Taylor & Francis Journals, vol. 52(36), pages 3919-3932, July.
    2. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    3. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    4. Robert A. Buckle & John Creedy, 2022. "Methods to evaluate institutional responses to performance‐based research funding systems," Australian Economic Papers, Wiley Blackwell, vol. 61(3), pages 615-634, September.
    5. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    6. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.
    7. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.
    8. Buckle, Robert A & Creedy, John, 2022. "The Performance Based Research Fund in NZ: Taking Stock and Looking Forward," Working Paper Series 21354, Victoria University of Wellington, Chair in Public Finance.
    9. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    10. Robert A. Buckle & John Creedy, 2020. "The ‘disciplinary effect’ of the performance-based research fund process in New Zealand," New Zealand Economic Papers, Taylor & Francis Journals, vol. 54(2), pages 107-126, May.
    11. Miketa, Asami & Mulder, Peter, 2005. "Energy productivity across developed and developing countries in 10 manufacturing sectors: Patterns of growth and convergence," Energy Economics, Elsevier, vol. 27(3), pages 429-453, May.
    12. Leone Leonida & Leone Leonida & Daniel Montolio, 2003. "Public Capital, Growth and Convergence in Spain. A Counterfactual Density Estimation Approach," Working Papers 2003/3, Institut d'Economia de Barcelona (IEB).
    13. Rachel Griffith & Stephen Redding & John Van Reenen, 2004. "Mapping the Two Faces of R&D: Productivity Growth in a Panel of OECD Industries," The Review of Economics and Statistics, MIT Press, vol. 86(4), pages 883-895, November.
    14. Paul Johnson & Chris Papageorgiou, 2020. "What Remains of Cross-Country Convergence?," Journal of Economic Literature, American Economic Association, vol. 58(1), pages 129-175, March.
    15. Giovanni Caggiano & Leone Leonida, 2013. "Multimodality in the distribution of GDP and the absolute convergence hypothesis," Empirical Economics, Springer, vol. 44(3), pages 1203-1215, June.
    16. Mario Coccia & Greta Falavigna & Alessandro Manello, 2015. "The impact of hybrid public and market-oriented financing mechanisms on the scientific portfolio and performances of public research labs: a scientometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 151-168, January.
    17. Rassekh, Farhad & Panik, Michael J. & Kolluri, Bharat R., 2001. "A test of the convergence hypothesis: the OECD experience, 1950-1990," International Review of Economics & Finance, Elsevier, vol. 10(2), pages 147-157.
    18. Brander, James A & Dowrick, Steve, 1994. "The Role of Fertility and Population in Economic Growth: Empirical Results from Aggregate Cross-National Data," Journal of Population Economics, Springer;European Society for Population Economics, vol. 7(1), pages 1-25.
    19. John Knight & Sai Ding, 2008. "Why has China Grown so Fast? The Role of Structural Change," Economics Series Working Papers 415, University of Oxford, Department of Economics.
    20. Charles I. Jones, 1997. "On the Evolution of the World Income Distribution," Journal of Economic Perspectives, American Economic Association, vol. 11(3), pages 19-36, Summer.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vuw:vuwcpf:20931. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Library Technology Services (email available below). General contact details of provider: https://edirc.repec.org/data/fcvuwnz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.