IDEAS home Printed from https://ideas.repec.org/p/sru/ssewps/2015-27.html
   My bibliography  Save this paper

Research Assessment in the UK and Italy: Costly and Difficult, But Probably Worth it (At Least For a While)

Author

Listed:
  • Aldo Geuna

    (Department of Economics
    Statistics Cognetti De Martiis, University of Torino)

  • Matteo Piolatto

    (Department of Social and Political Studies, University of Milan)

Abstract

This paper provides a comparative analysis of the development of the UK and Italian university research funding systems with a special focus on Peer Review-Based Research Assessment (PRBRA) and its cost. Much of the debate surrounding the value of performance-based allocation systems hinges on the disadvantages versus the benefits of their implementation, and there is very little evidence on either their absolute cost or their cost relative to other allocation systems. Our objective is to fill this gap, collating the best possible estimates of the costs of alternative research funding methods to inform the ongoing policy debate. First, we compare funding in the UK and Italy during the period 2005-2012 and analyze the development of performance-based allocation in the two systems. Second, based on public reports and documents collected from universities, we discuss the public agency and university costs of RAE2008 and REF2014 and provide some estimates for VQR2012. We find that RAE2008 costs accounted for less than 1% of the total performance allocation in the related period while the VQR2012 efficiency ratio is estimated at around 2.5%. Finally, we compare the costs and efficiency ratios of PRBRA with metrics-based assessment and Research Council allocations and show that costs increase going from metrics to PRBRA to Research Council allocation.

Suggested Citation

  • Aldo Geuna & Matteo Piolatto, 2015. "Research Assessment in the UK and Italy: Costly and Difficult, But Probably Worth it (At Least For a While)," SPRU Working Paper Series 2015-27, SPRU - Science Policy Research Unit, University of Sussex Business School.
    • Handle: RePEc:sru:ssewps:2015-27
    as

    Download full text from publisher

    File URL: http://www.sussex.ac.uk/spru/documents/2015-27-swps-geuna-piolatto.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Savona, María, 2016. "Global structural change and value chains in services: a reappraisal," Coediciones, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL), number 43200, March.
    2. Aldo Geuna, 2001. "The Changing Rationale for European University Research Funding: Are There Negative Unintended Consequences?," Journal of Economic Issues, Taylor & Francis Journals, vol. 35(3), pages 607-632, September.
    3. Aldo Geuna, 1999. "The Economics of Knowledge Production," Books, Edward Elgar Publishing, number 1689.
    4. Mariana Mazzucato, 2015. "From Market Fixing to Market-Creating: A New Framework for Economic Policy," SPRU Working Paper Series 2015-25, SPRU - Science Policy Research Unit, University of Sussex Business School.
    5. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    6. Mariana Mazzucato & Caetano Penna, 2015. "The Rise of Mission-Oriented State Investment Banks: The Cases of Germany’s KfW and Brazil’s BNDES," SPRU Working Paper Series 2015-26, SPRU - Science Policy Research Unit, University of Sussex Business School.
    7. Alessio Ancaiani & Alberto F. Anfossi & Anna Barbara & Sergio Benedetto & Brigida Blasi & Valentina Carletti & Tindaro Cicero & Alberto Ciolfi & Filippo Costa & Giovanna Colizza & Marco Costantini & F, 2015. "Evaluating scientific research in Italy: The 2004–10 research evaluation exercise," Research Evaluation, Oxford University Press, vol. 24(3), pages 242-255.
    8. Paula Stephan, 2014. "The Endless Frontier: Reaping What Bush Sowed?," NBER Chapters, in: The Changing Frontier: Rethinking Science and Innovation Policy, pages 321-366, National Bureau of Economic Research, Inc.
    9. Javier López González & Valentina Meliciani & Maria Savona, 2019. "When Linder meets Hirschman: inter-industry linkages and global value chains in business services," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 28(6), pages 1555-1586.
    10. Aldo Geuna & Federica Rossi, 2015. "The University and the Economy," Books, Edward Elgar Publishing, number 15528.
    11. repec:elg:eechap:15528_6 is not listed on IDEAS
    12. Ben R Martin, 2011. "The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster?," Research Evaluation, Oxford University Press, vol. 20(3), pages 247-254, September.
    13. Lee Stapleton, 2015. "Do academics doubt their own research?," SPRU Working Paper Series 2015-24, SPRU - Science Policy Research Unit, University of Sussex Business School.
    14. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    15. J. Sylvan Katz, 2015. "What is a Complex Innovation System?," SPRU Working Paper Series 2015-21, SPRU - Science Policy Research Unit, University of Sussex Business School.
    16. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    17. Tommaso Ciarli & Alex Coad & Ismael Rafols, 2015. "Quantitative Analysis of Technology Futures: A review of Techniques, Uses and Characteristics," SPRU Working Paper Series 2015-23, SPRU - Science Policy Research Unit, University of Sussex Business School.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lorenz Gollwitzer & David Ockwell & Adrian Ely, 2015. "Institutional Innovation in the Management of Pro-Poor Energy Access in East Africa," SPRU Working Paper Series 2015-29, SPRU - Science Policy Research Unit, University of Sussex Business School.
    2. Mariana Mazzucato, 2015. "The Green Entrepreneurial State," SPRU Working Paper Series 2015-28, SPRU - Science Policy Research Unit, University of Sussex Business School.
    3. Friedemann Polzin & Steve Sorrell & Colin Nolden, 2015. "Innovative Procurement Frameworks for Energy Performance Contracting in the UK Public Sector," SPRU Working Paper Series 2015-31, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Mariana Mazzucato, 2015. "From Market Fixing to Market-Creating: A New Framework for Economic Policy," SPRU Working Paper Series 2015-25, SPRU - Science Policy Research Unit, University of Sussex Business School.
    5. Geuna, Aldo & Piolatto, Matteo, 2014. "The Development of Research Assessment in the UK and Italy: Costly and difficult, but probably worth (for a while)"," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201416, University of Turin.
    6. Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    7. Civera, Alice & Lehmann, Erik E. & Paleari, Stefano & Stockinger, Sarah A.E., 2020. "Higher education policy: Why hope for quality when rewarding quantity?," Research Policy, Elsevier, vol. 49(8).
    8. Wang, Jian & Lee, You-Na & Walsh, John P., 2018. "Funding model and creativity in science: Competitive versus block funding and status contingency effects," Research Policy, Elsevier, vol. 47(6), pages 1070-1083.
    9. Lawson, Cornelia & Geuna, Aldo & Finardi, Ugo, 2021. "The funding-productivity-gender nexus in science, a multistage analysis," Research Policy, Elsevier, vol. 50(3).
    10. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    11. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    12. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.
    13. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    14. Gustavo Crespi & Aldo Geuna & Lionel Nesta, 2007. "The mobility of university inventors in Europe," The Journal of Technology Transfer, Springer, vol. 32(3), pages 195-215, June.
    15. Robert A. Buckle & John Creedy & Ashley Ball, 2021. "Fifteen Years of a PBRFS in New Zealand: Incentives and Outcomes," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 54(2), pages 208-230, June.
    16. Giliberto Capano & Benedetto Lepori, 2024. "Designing policies that could work: understanding the interaction between policy design spaces and organizational responses in public sector," Policy Sciences, Springer;Society of Policy Sciences, vol. 57(1), pages 53-82, March.
    17. Benedetto Lepori & Aldo Geuna & Valerio Veglio, 2017. "A Typology of European Research Universities. Differentiation, Layering and Resource Distribution," SPRU Working Paper Series 2017-01, SPRU - Science Policy Research Unit, University of Sussex Business School.
    18. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    19. Gustavo Crespi & Aldo Geuna, 2005. "Modelling and Measuring Scientific Production: Results for a Panel of OECD Countries," SPRU Working Paper Series 133, SPRU - Science Policy Research Unit, University of Sussex Business School.
    20. Migheli, Matteo & Zotti, Roberto, 2020. "The strange case of the Matthew effect and beauty contests: Research evaluation and specialisation in Italian universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).

    More about this item

    Keywords

    Higher Education Policy; Public Funding; Research Assessment; University Research challenges; public finance; financialisation; innovation;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • Z18 - Other Special Topics - - Cultural Economics - - - Public Policy
    • H5 - Public Economics - - National Government Expenditures and Related Policies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sru:ssewps:2015-27. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: University of Sussex Business School Communications Team (email available below). General contact details of provider: https://edirc.repec.org/data/spessuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.