IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v40y2011i10p1369-1379.html
   My bibliography  Save this article

‘Impact’ in the proposals for the UK's Research Excellence Framework: Shifting the boundaries of academic autonomy

Author

Listed:
  • Smith, Simon
  • Ward, Vicky
  • House, Allan

Abstract

Evaluation of university-based research already has a reasonably long tradition in the UK, but proposals to revise the framework for national evaluation aroused controversy in the academic community because they envisage assessing more explicitly than before the economic, social and cultural ‘impact’ of research as well as its scientific quality. Using data from the 2009 public consultation on the proposals for a Research Excellence Framework, this paper identifies three main lines of controversy: the threats to academic autonomy implied in the definition of expert review and the delimitation of reviewers, the scope for boundary-work in the construction of impact narratives and case studies, and the framing of knowledge translation by the stipulation that impact ‘builds on’ research. Given the behaviour-shaping effects of research evaluation, the paper demonstrates how the proposed changes could help embed impact considerations among the routine reflexive tools of university researchers and enhance rather than restrict academic autonomy at the level of research units. It also argues that the REF could constitute an important dialogical space for negotiating science–society relations in an era of increasing heteronomy between academia, state and industry. But the paper raises doubts about whether the proposed operationalisation of impact is adequate to evaluate the ways that research and knowledge translation are actually carried out.

Suggested Citation

  • Smith, Simon & Ward, Vicky & House, Allan, 2011. "‘Impact’ in the proposals for the UK's Research Excellence Framework: Shifting the boundaries of academic autonomy," Research Policy, Elsevier, vol. 40(10), pages 1369-1379.
  • Handle: RePEc:eee:respol:v:40:y:2011:i:10:p:1369-1379
    DOI: 10.1016/j.respol.2011.05.026
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733311001077
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.respol.2011.05.026?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Katharine Barker, 2007. "The UK Research Assessment Exercise: the evolution of a national research evaluation system," Research Evaluation, Oxford University Press, vol. 16(1), pages 3-12, March.
    2. Claire Donovan, 2009. "Gradgrinding the Social Sciences: The Politics of Metrics of Political Science," Political Studies Review, Political Studies Association, vol. 7(1), pages 73-83, January.
    3. Grit Laudel, 2006. "Conclave in the Tower of Babel: how peers review interdisciplinary research proposals," Research Evaluation, Oxford University Press, vol. 15(1), pages 57-68, April.
    4. Loet Leydesdorff & Henry Etzkowitz, 1996. "Emergence of a Triple Helix of university—industry—government relations," Science and Public Policy, Oxford University Press, vol. 23(5), pages 279-286, October.
    5. Claire Donovan, 2007. "Introduction: Future pathways for science policy and research assessment: Metrics vs peer review, quality vs impact," Science and Public Policy, Oxford University Press, vol. 34(8), pages 538-542, October.
    6. Linda Butler, 2007. "Assessing university research: A plea for a balanced approach," Science and Public Policy, Oxford University Press, vol. 34(8), pages 565-574, October.
    7. Linda Butler & Ian McAllister, 2009. "Metrics or Peer Review? Evaluating the 2001 UK Research Assessment Exercise in Political Science," Political Studies Review, Political Studies Association, vol. 7(1), pages 3-17, January.
    8. Heinze, Thomas & Shapira, Philip & Rogers, Juan D. & Senker, Jacqueline M., 2009. "Organizational and institutional influences on creativity in scientific research," Research Policy, Elsevier, vol. 38(4), pages 610-623, May.
    9. Benner, Mats & Sandstrom, Ulf, 2000. "Institutionalizing the triple helix: research funding and norms in the academic system," Research Policy, Elsevier, vol. 29(2), pages 291-301, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Belkhouja, Mustapha & Fattoum, Senda & Yoon, Hyungseok (David), 2021. "Does greater diversification increase individual productivity? The moderating effect of attention allocation," Research Policy, Elsevier, vol. 50(6).
    2. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    3. Richard McManus & Karen Mumford & Cristina Sechel, 2017. "The Selection of Economics Lecturers into the 2014 UK Research Excellence Framework Exercise: Outputs and Gender," Discussion Papers 17/16, Department of Economics, University of York.
    4. Salter, Ammon & Salandra, Rossella & Walker, James, 2017. "Exploring preferences for impact versus publications among UK business and management academics," Research Policy, Elsevier, vol. 46(10), pages 1769-1782.
    5. Shortt, Niamh K. & Pearce, Jamie & Mitchell, Richard & Smith, Katherine E., 2016. "Taking health geography out of the academy: Measuring academic impact," Social Science & Medicine, Elsevier, vol. 168(C), pages 265-272.
    6. Ming-Jer Chen, 2018. "Scholarship-practice “oneness” of an academic career: The entrepreneurial pursuit of an expansive view of management scholarship," Asia Pacific Journal of Management, Springer, vol. 35(4), pages 859-886, December.
    7. Huang, Haijie & Lee, Edward & Lyu, Changjiang & Zhu, Zhenmei, 2016. "The effect of accounting academics in the boardroom on the value relevance of financial reporting information," International Review of Financial Analysis, Elsevier, vol. 45(C), pages 18-30.
    8. Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    9. Pablo D’Este & Irene Ramos-Vielba & Richard Woolley & Nabil Amara, 2018. "How do researchers generate scientific and societal impacts? Toward an analytical and operational framework," Science and Public Policy, Oxford University Press, vol. 45(6), pages 752-763.
    10. Irwin Feller, 2013. "Peer review and expert panels as techniques for evaluating the quality of academic research," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 5, pages 115-142, Edward Elgar Publishing.
    11. Richard McManus & Karen Mumford & Cristina Sechel, 2022. "Measuring research excellence amongst economics lecturers in the UK," Bulletin of Economic Research, Wiley Blackwell, vol. 74(2), pages 386-404, April.
    12. Gobinda Chowdhury & Kushwanth Koya & Pete Philipson, 2016. "Measuring the Impact of Research: Lessons from the UK’s Research Excellence Framework 2014," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-15, June.
    13. Michael C. Calver & Maggie Lilith & Christopher R. Dickman, 2013. "A ‘perverse incentive’ from bibliometrics: could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 243-255, April.
    14. Martin Meyer & Kevin Grant & Piera Morlacchi & Dagmara Weckowska, 2014. "Triple Helix indicators as an emergent area of enquiry: a bibliometric perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(1), pages 151-174, April.
    15. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beesley, Lisa G. A., 2003. "Science policy in changing times: are governments poised to take full advantage of an institution in transition?," Research Policy, Elsevier, vol. 32(8), pages 1519-1531, September.
    2. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    3. Matthew L. Wallace & Ismael Rafols, 2016. "Shaping the Agenda of a Grand Challenge: Institutional Mediation of Priorities in Avian Influenza Research," SPRU Working Paper Series 2016-02, SPRU - Science Policy Research Unit, University of Sussex Business School.
    4. Wallace, Matthew L. & Ràfols, Ismael, 2018. "Institutional shaping of research priorities: A case study on avian influenza," Research Policy, Elsevier, vol. 47(10), pages 1975-1989.
    5. Pablo D’Este & Puay Tang & Surya Mahdi & Andy Neely & Mabel Sánchez-Barrioluengo, 2013. "The pursuit of academic excellence and business engagement: is it irreconcilable?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 481-502, May.
    6. J. W. Fedderke, 2013. "The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 177-206, November.
    7. O’Kane, Conor, 2018. "Technology transfer executives' backwards integration: An examination of interactions between university technology transfer executives and principal investigators," Technovation, Elsevier, vol. 76, pages 64-77.
    8. Thai Thi Minh & Carsten Nico Hjotrsø, 2015. "Relational dynamics in the multi-helices knowledge production system: A new institutionalism perspective," Globelics Working Paper Series 2015-08, Globelics - Global Network for Economics of Learning, Innovation, and Competence Building Systems, Aalborg University, Department of Business and Management.
    9. M. Teresa Antonio-García & Irene López-Navarro & Jesús Rey-Rocha, 2014. "Determinants of success for biomedical researchers: a perception-based study in a health science research environment," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1747-1779, December.
    10. Diego Chavarro & Puay Tang & Ismael Rafols, 2014. "Interdisciplinarity and research on local issues: evidence from a developing country," Research Evaluation, Oxford University Press, vol. 23(3), pages 195-209.
    11. Elvira Uyarra & Jens Sörvik & Inger Midtkandal, 2014. "Inter-regional Collaboration in Research and Innovation Strategies for Smart Specialisation (RIS3). S3 Working Paper Series no 6/2014," JRC Research Reports JRC91963, Joint Research Centre.
    12. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    13. Burmaoglu, Serhat & Sartenaer, Olivier & Porter, Alan, 2019. "Conceptual definition of technology emergence: A long journey from philosophy of science to science policy," Technology in Society, Elsevier, vol. 59(C).
    14. Alberto Anfossi & Alberto Ciolfi & Filippo Costa & Giorgio Parisi & Sergio Benedetto, 2016. "Large-scale assessment of research outputs through a weighted combination of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 671-683, May.
    15. Svein Kyvik & Ingvild Reymert, 2017. "Research collaboration in groups and networks: differences across academic fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 951-967, November.
    16. Munari, Federico & Sobrero, Maurizio & Toschi, Laura, 2018. "The university as a venture capitalist? Gap funding instruments for technology transfer," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 70-84.
    17. Leona Yi-Fan Su & Michael A. Cacciatore & Dominique Brossard & Elizabeth A. Corley & Dietram A. Scheufele & Michael A. Xenos, 2016. "Attitudinal gaps: How experts and lay audiences form policy attitudes toward controversial science," Science and Public Policy, Oxford University Press, vol. 43(2), pages 196-206.
    18. Barry Bozeman & Gordon Kingsley, 2013. "Research value mapping and evaluation: theory and application," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 7, pages 166-189, Edward Elgar Publishing.
    19. Li, Yin & Arora, Sanjay & Youtie, Jan & Shapira, Philip, 2018. "Using web mining to explore Triple Helix influences on growth in small and mid-size firms," Technovation, Elsevier, vol. 76, pages 3-14.
    20. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:40:y:2011:i:10:p:1369-1379. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.