IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v42y2013i9p1657-1666.html
   My bibliography  Save this article

The UK and Italian research assessment exercises face to face

Author

Listed:
  • Rebora, Gianfranco
  • Turri, Matteo

Abstract

In the UK the Research Assessment Exercise (RAE) is being replaced by the Research Excellence Framework (REF); similarly, in Italy the former Research Evaluation Exercise (VTR 2001–2003) has been revised and extended and a new cycle of assessment was activated in Autumn 2011 (VQR 2004–2010). The analysis of the development and the consequences of both British and Italian assessment exercises is carried out on the basis of three theoretical frameworks – new public management (NPM), neo-institutional sociology (NIS), and organisational control theory (OCT). This last framework is crucial because the control systems have an effective sense-making capacity and it also exposes a different awareness about control systems in the British and Italian experiences.

Suggested Citation

  • Rebora, Gianfranco & Turri, Matteo, 2013. "The UK and Italian research assessment exercises face to face," Research Policy, Elsevier, vol. 42(9), pages 1657-1666.
    • Handle: RePEc:eee:respol:v:42:y:2013:i:9:p:1657-1666
    DOI: 10.1016/j.respol.2013.06.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733313001133
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2013.06.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anthony Hopwood, 2008. "Changing Pressures on the Research Process: On Trying to Research in an Age when Curiosity is not Enough," European Accounting Review, Taylor & Francis Journals, vol. 17(1), pages 87-96.
    2. Jane Broadbent, 2010. "The UK Research Assessment Exercise: Performance Measurement and Resource Allocation," Australian Accounting Review, CPA Australia, vol. 20(1), pages 14-23, March.
    3. Katharine Barker, 2007. "The UK Research Assessment Exercise: the evolution of a national research evaluation system," Research Evaluation, Oxford University Press, vol. 16(1), pages 3-12, March.
    4. Smith, Simon & Ward, Vicky & House, Allan, 2011. "‘Impact’ in the proposals for the UK's Research Excellence Framework: Shifting the boundaries of academic autonomy," Research Policy, Elsevier, vol. 40(10), pages 1369-1379.
    5. Henk F. Moed, 2008. "UK Research Assessment Exercises: Informed judgments on research quality or quantity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 153-161, January.
    6. Kathleen M. Eisenhardt, 1985. "Control: Organizational and Economic Approaches," Management Science, INFORMS, vol. 31(2), pages 134-149, February.
    7. Laura B. Cardinal, 2001. "Technological Innovation in the Pharmaceutical Industry: The Use of Organizational Control in Managing Research and Development," Organization Science, INFORMS, vol. 12(1), pages 19-36, February.
    8. Flamholtz, Eric G. & Das, T. K. & Tsui, Anne S., 1985. "Toward an integrative framework of organizational control," Accounting, Organizations and Society, Elsevier, vol. 10(1), pages 35-50, January.
    9. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    10. Emanuela Reale & Anna Barbara & Antonio Costantini, 2007. "Peer review for the evaluation of academic research: lessons from the Italian experience," Research Evaluation, Oxford University Press, vol. 16(3), pages 216-228, September.
    11. Pollitt, Christopher & Bouckaert, Geert, 2004. "Public Management Reform: A Comparative Analysis," OUP Catalogue, Oxford University Press, edition 2, number 9780199268498, Decembrie.
    12. Langfield-Smith, Kim, 1997. "Management control systems and strategy: A critical review," Accounting, Organizations and Society, Elsevier, vol. 22(2), pages 207-232, February.
    13. Johnson, William H.A., 2011. "Managing university technology development using organizational control theory," Research Policy, Elsevier, vol. 40(6), pages 842-852, July.
    14. Fagerberg, Jan & Landström, Hans & Martin, Ben R., 2012. "Exploring the emerging knowledge base of ‘the knowledge society’," Research Policy, Elsevier, vol. 41(7), pages 1121-1131.
    15. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    16. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    17. Parker, Lee, 2011. "University corporatisation: Driving redefinition," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 22(4), pages 434-450.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bagues, Manuel & Sylos-Labini, Mauro & Zinovyeva, Natalia, 2019. "A walk on the wild side: ‘Predatory’ journals and information asymmetries in scientific evaluations," Research Policy, Elsevier, vol. 48(2), pages 462-477.
    2. Belkhouja, Mustapha & Fattoum, Senda & Yoon, Hyungseok (David), 2021. "Does greater diversification increase individual productivity? The moderating effect of attention allocation," Research Policy, Elsevier, vol. 50(6).
    3. Tharapos, Meredith & Marriott, Neil, 2020. "Beauty is in the eye of the beholder: Research quality in accounting education," The British Accounting Review, Elsevier, vol. 52(5).
    4. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    5. Reed, M.S. & Ferré, M. & Martin-Ortega, J. & Blanche, R. & Lawford-Rolfe, R. & Dallimer, M. & Holden, J., 2021. "Evaluating impact from research: A methodological framework," Research Policy, Elsevier, vol. 50(4).
    6. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    7. Francesco Giovanni Avallone & Alberto Quagli & Paola Ramassa, 2022. "Interdisciplinary research by accounting scholars: An exploratory study," FINANCIAL REPORTING, FrancoAngeli Editore, vol. 2022(2), pages 5-34.
    8. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    9. Salter, Ammon & Salandra, Rossella & Walker, James, 2017. "Exploring preferences for impact versus publications among UK business and management academics," Research Policy, Elsevier, vol. 46(10), pages 1769-1782.
    10. John Rigby & Barbara Jones, 2020. "Bringing the doctoral thesis by published papers to the Social Sciences and the Humanities: A quantitative easing? A small study of doctoral thesis submission rules and practice in two disciplines in ," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1387-1409, August.
    11. Jappelli, Tullio & Nappi, Carmela Anna & Torrini, Roberto, 2017. "Gender effects in research evaluation," Research Policy, Elsevier, vol. 46(5), pages 911-924.
    12. Camil Demetrescu & Andrea Ribichini & Marco Schaerf, 2020. "Are Italian research assessment exercises size-biased?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 533-549, October.
    13. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    14. Pilonato, Silvia & Monfardini, Patrizio, 2020. "Performance measurement systems in higher education: How levers of control reveal the ambiguities of reforms," The British Accounting Review, Elsevier, vol. 52(3).
    15. Stefano Bianchini & Francesco Lissoni & Michele Pezzoni & Lorenzo Zirulia, 2016. "The economics of research, consulting, and teaching quality: theory and evidence from a technical university," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 25(7), pages 668-691, October.
    16. Civera, Alice & Lehmann, Erik E. & Paleari, Stefano & Stockinger, Sarah A.E., 2020. "Higher education policy: Why hope for quality when rewarding quantity?," Research Policy, Elsevier, vol. 49(8).
    17. Biancardi, Daniele & Bratti, Massimiliano, 2018. "The Effect of the First Italian Research Evaluation Exercise on Student Enrolment Choices," IZA Discussion Papers 11302, Institute of Labor Economics (IZA).
    18. Mehmet Pinar & Timothy J Horne, 2022. "Assessing research excellence: Evaluating the Research Excellence Framework," Research Evaluation, Oxford University Press, vol. 31(2), pages 173-187.
    19. Marco Seeber & Jef Vlegels & Mattia Cattaneo, 2022. "Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(8), pages 1106-1126, August.
    20. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.
    21. Cinzia Daraio, 2017. "A framework for the Assessment of Research and its impacts," DIAG Technical Reports 2017-04, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    22. Biancardi, Daniele & Bratti, Massimiliano, 2019. "The effect of introducing a Research Evaluation Exercise on student enrolment: Evidence from Italy," Economics of Education Review, Elsevier, vol. 69(C), pages 73-93.
    23. Demetrescu, Camil & Lupia, Francesco & Mendicelli, Angelo & Ribichini, Andrea & Scarcello, Francesco & Schaerf, Marco, 2019. "On the Shapley value and its application to the Italian VQR research assessment exercise," Journal of Informetrics, Elsevier, vol. 13(1), pages 87-104.
    24. Calogero Guccio & Marco Ferdinando Martorana & Isidoro Mazza, 2016. "Efficiency assessment and convergence in teaching and research in Italian public universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1063-1094, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    2. Sebastian Goebel & Barbara E. Weißenberger, 2017. "Effects of management control mechanisms: towards a more comprehensive analysis," Journal of Business Economics, Springer, vol. 87(2), pages 185-219, February.
    3. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 9447, Victoria University of Wellington, Chair in Public Finance.
    4. Sandeep Rustagi & William R. King & Laurie J. Kirsch, 2008. "Predictors of Formal Control Usage in IT Outsourcing Partnerships," Information Systems Research, INFORMS, vol. 19(2), pages 126-143, June.
    5. Degl’Innocenti, Marta & Matousek, Roman & Tzeremes, Nickolaos G., 2019. "The interconnections of academic research and universities’ “third mission”: Evidence from the UK," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    6. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    7. Rauter, Romana & Globocnik, Dietfried & Baumgartner, Rupert J., 2023. "The role of organizational controls to advance sustainability innovation performance," Technovation, Elsevier, vol. 128(C).
    8. Geuna, Aldo & Piolatto, Matteo, 2016. "Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)," Research Policy, Elsevier, vol. 45(1), pages 260-271.
    9. Robert A. Buckle & John Creedy, 2022. "Methods to evaluate institutional responses to performance‐based research funding systems," Australian Economic Papers, Wiley Blackwell, vol. 61(3), pages 615-634, September.
    10. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    11. Emil Inauen & Margit Osterloh & Bruno Frey & Fabian Homberg, 2015. "How a multiple orientation of control reduces governance failures: a focus on monastic auditing," Journal of Management & Governance, Springer;Accademia Italiana di Economia Aziendale (AIDEA), vol. 19(4), pages 763-796, November.
    12. Michael C. Calver & Maggie Lilith & Christopher R. Dickman, 2013. "A ‘perverse incentive’ from bibliometrics: could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 243-255, April.
    13. Buckle, Robert A. & Creedy, John & Ball, Ashley, 2020. "A Schumpeterian Gale: Using Longitudinal Data to Evaluate Responses to Performance-Based Research Funding Systems," Working Paper Series 21104, Victoria University of Wellington, Chair in Public Finance.
    14. Belkhouja, Mustapha & Fattoum, Senda & Yoon, Hyungseok (David), 2021. "Does greater diversification increase individual productivity? The moderating effect of attention allocation," Research Policy, Elsevier, vol. 50(6).
    15. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    16. Hartmann, Frank G. H., 2000. "The appropriateness of RAPM: toward the further development of theory," Accounting, Organizations and Society, Elsevier, vol. 25(4-5), pages 451-482, May.
    17. Martin-Sardesai, Ann & Irvine, Helen & Tooley, Stuart & Guthrie, James, 2017. "Organizational change in an Australian university: Responses to a research assessment exercise," The British Accounting Review, Elsevier, vol. 49(4), pages 399-412.
    18. Robert A. Buckle & John Creedy & Norman Gemmell, 2022. "Sources of convergence and divergence in university research quality: evidence from the performance-based research funding system in New Zealand," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3021-3047, June.
    19. Gavin M Schwarz & Karin Sanders & Dave Bouckenooghe, 2020. "In the driving seat: Executive’s perceived control over environment," Australian Journal of Management, Australian School of Business, vol. 45(2), pages 317-342, May.
    20. Alberto Anfossi & Alberto Ciolfi & Filippo Costa & Giorgio Parisi & Sergio Benedetto, 2016. "Large-scale assessment of research outputs through a weighted combination of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 671-683, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:42:y:2013:i:9:p:1657-1666. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.