IDEAS home Printed from https://ideas.repec.org/a/kap/jtecht/v47y2022i3d10.1007_s10961-017-9608-6.html
   My bibliography  Save this article

Learning from the UK’s research impact assessment exercise: a case study of a retrospective impact assessment exercise and questions for the future

Author

Listed:
  • Molly Morgan Jones

    (RAND Europe)

  • Catriona Manville

    (RAND Europe)

  • Joanna Chataway

    (University of Sussex)

Abstract

National governments spend significant amounts of money supporting public research. However, in an era where the international economic climate has led to budget cuts, policymakers increasingly are looking to justify the returns from public investments, including in science and innovation. The so-called ‘impact agenda’ which has emerged in many countries around the world is part of this response; an attempt to understand and articulate for the public what benefits arise from the research that is funded. The United Kingdom is the most progressed in implementing this agenda and in 2014 the national research assessment exercise, the Research Excellence Framework, for the first time included the assessment of research impact as a component. For the first time within a dual funding system, funding would be awarded not only on the basis of the academic quality of research, but also on the wider impacts of that research. In this paper we outline the context and approach taken by the UK government, along with some of the core challenges that exist in implementing such an exercise. We then synthesise, together for the first time, the results of the only two national evaluations of the exercise and offer reflections for future exercises both in the UK and internationally.

Suggested Citation

  • Molly Morgan Jones & Catriona Manville & Joanna Chataway, 2022. "Learning from the UK’s research impact assessment exercise: a case study of a retrospective impact assessment exercise and questions for the future," The Journal of Technology Transfer, Springer, vol. 47(3), pages 722-746, June.
  • Handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9608-6
    DOI: 10.1007/s10961-017-9608-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10961-017-9608-6
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10961-017-9608-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Annette Boaz & Siobhan Fitzpatrick & Ben Shaw, 2009. "Assessing the impact of research on policy: A literature review," Science and Public Policy, Oxford University Press, vol. 36(4), pages 255-270, May.
    2. Jordi Molas-Gallart & Puay Tang, 2011. "Tracing ‘productive interactions’ to identify social impacts: an example from the social sciences," Research Evaluation, Oxford University Press, vol. 20(3), pages 219-226, September.
    3. Christopher Freeman, 2008. "Systems of Innovation," Books, Edward Elgar Publishing, number 12797.
    4. Martin, Ben R. & Irvine, John, 1993. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
    5. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    6. Lindell Bromham & Russell Dinnage & Xia Hua, 2016. "Interdisciplinary research has consistently lower funding success," Nature, Nature, vol. 534(7609), pages 684-687, June.
    7. Steven Hill, 2016. "Assessing (for) impact: future assessment of the societal impact of research," Palgrave Communications, Palgrave Macmillan, vol. 2(1), pages 1-7, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sabrina Petersohn & Thomas Heinze, 2018. "Professionalization of bibliometric research assessment. Insights from the history of the Leiden Centre for Science and Technology Studies (CWTS)," Science and Public Policy, Oxford University Press, vol. 45(4), pages 565-578.
    2. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2014. "Assessing national strengths and weaknesses in research fields," Journal of Informetrics, Elsevier, vol. 8(3), pages 766-775.
    3. van den Besselaar, Peter & Heyman, Ulf & Sandström, Ulf, 2017. "Perverse effects of output-based research funding? Butler’s Australian case revisited," Journal of Informetrics, Elsevier, vol. 11(3), pages 905-918.
    4. Ohid Yaqub & Dmitry Malkov & Josh Siepel, 2023. "How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework," Research Evaluation, Oxford University Press, vol. 32(2), pages 273-285.
    5. Helka Kalliomäki & Sampo Ruoppila & Jenni Airaksinen, 2021. "It takes two to tango: Examining productive interactions in urban research collaboration [Generating Research Questions through Problematization]," Research Evaluation, Oxford University Press, vol. 30(4), pages 529-539.
    6. D’Este, Pablo & Robinson-García, Nicolás, 2023. "Interdisciplinary research and the societal visibility of science: The advantages of spanning multiple and distant scientific fields," Research Policy, Elsevier, vol. 52(2).
    7. Rafols, Ismael & Stirling, Andy, 2020. "Designing indicators for opening up evaluation. Insights from research assessment," SocArXiv h2fxp, Center for Open Science.
    8. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.
    9. Alis Oancea, 2019. "Research governance and the future(s) of research assessment," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-12, December.
    10. Owen, Richard & Pansera, Mario & Macnaghten, Phil & Randles, Sally, 2021. "Organisational institutionalisation of responsible innovation," Research Policy, Elsevier, vol. 50(1).
    11. Henk F. Moed & Gali Halevi, 2015. "Multidimensional assessment of scholarly research impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 1988-2002, October.
    12. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    13. A. Gaunand & L. Colinet & P.-B. Joly & M. Matt, 2022. "Counting what really counts? Assessing the political impact of science," The Journal of Technology Transfer, Springer, vol. 47(3), pages 699-721, June.
    14. Kroll, Henning & Hansmeier, Hendrik & Hufnagl, Miriam, 2022. "Productive interactions in basic research an enquiry into impact pathways at the DESY synchrotron," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    15. Rousseau, Sandra & Catalano, Giuseppe & Daraio, Cinzia, 2021. "Can we estimate a monetary value of scientific publications?," Research Policy, Elsevier, vol. 50(1).
    16. Cinzia Daraio, 2017. "A framework for the Assessment of Research and its impacts," DIAG Technical Reports 2017-04, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    17. Kanayo K. Ogujiuba & Maria Eggink & Ebenezer Olamide, 2023. "Interaction and Main Effects of Finance Support and Other Business Support Services on the Entrepreneurial Ecosystem: A Case Study of the Mpumalanga Province, South Africa," Economies, MDPI, vol. 11(6), pages 1-23, May.
    18. Roger Hayter & Klaus Edenhoffer, 2016. "Evolutionary Geography of a Mature Resource Sector: Shakeouts and Shakeins in British Columbia's Forest Industries 1980 to 2008," Growth and Change, Wiley Blackwell, vol. 47(4), pages 497-519, December.
    19. Jill Johnes, 2018. "University rankings: What do they really show?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 585-606, April.
    20. Henk F. Moed, 2000. "Bibliometric Indicators Reflect Publication and Management Strategies," Scientometrics, Springer;Akadémiai Kiadó, vol. 47(2), pages 323-346, February.

    More about this item

    Keywords

    Research impact; Research policy; Research Excellence Framework; Science of science;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • O3 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jtecht:v:47:y:2022:i:3:d:10.1007_s10961-017-9608-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.