Advanced Search
MyIDEAS: Login

How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management

Contents:

Author Info

  • Rafols, Ismael
  • Leydesdorff, Loet
  • O’Hare, Alice
  • Nightingale, Paul
  • Stirling, Andy

Abstract

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management Schools (BMS) in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management Schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management Schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.sciencedirect.com/science/article/pii/S0048733312000765
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal Research Policy.

Volume (Year): 41 (2012)
Issue (Month): 7 ()
Pages: 1262-1282

as in new window
Handle: RePEc:eee:respol:v:41:y:2012:i:7:p:1262-1282

Contact details of provider:
Web page: http://www.elsevier.com/locate/respol

Related research

Keywords: Interdisciplinary; Evaluation; Ranking; Innovation; Bibliometrics; Research assessment;

Other versions of this item:

Find related papers by JEL classification:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Jan Fagerberg & Koson Sapprasert, 2010. "Innovation: Exploring the knowledge base," Working Papers on Innovation Studies 20100616, Centre for Technology, Innovation and Culture, University of Oslo.
  2. Fagerberg, Jan & Verspagen, Bart, 2009. "Innovation studies--The emerging structure of a new scientific field," Research Policy, Elsevier, vol. 38(2), pages 218-233, March.
  3. Huutoniemi, Katri & Klein, Julie Thompson & Bruun, Henrik & Hukkinen, Janne, 2010. "Analyzing interdisciplinarity: Typology and indicators," Research Policy, Elsevier, vol. 39(1), pages 79-88, February.
  4. Kiss, Istvan Z. & Broom, Mark & Craze, Paul G. & Rafols, Ismael, 2010. "Can epidemic models describe the diffusion of topics across disciplines?," Journal of Informetrics, Elsevier, vol. 4(1), pages 74-82.
  5. van Rijnsoever, Frank J. & Hessels, Laurens K., 2011. "Factors associated with disciplinary and interdisciplinary research collaboration," Research Policy, Elsevier, vol. 40(3), pages 463-472, April.
  6. Luis Sanz-Menéndez & Mar�a Bordons & M Angeles Zulueta, 2001. "Interdisciplinarity as a multidimensional concept: its measure in three different research areas," Research Evaluation, Oxford University Press, vol. 10(1), pages 47-58, April.
  7. Oswald, Andrew J., 2006. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-makers," The Warwick Economics Research Paper Series (TWERPS) 744, University of Warwick, Department of Economics.
  8. Clausen, Tommy & Fagerberg, Jan & Gulbrandsen, Magnus, 2012. "Mobilizing for change: A study of research units in emerging scientific fields," Research Policy, Elsevier, vol. 41(7), pages 1249-1261.
  9. Ismael Rafols & Martin Meyer, 2008. "Diversity and network coherence as indicators of interdisciplinarity: Case studies in bionanoscience," SPRU Working Paper Series 167, SPRU - Science and Technology Policy Research, University of Sussex.
  10. Jan Youtie & Maurizio Iacopetta & Stuart Graham, 2008. "Assessing the nature of nanotechnology: can we uncover an emerging general purpose technology?," The Journal of Technology Transfer, Springer, vol. 33(3), pages 315-329, June.
  11. Leydesdorff, Loet & Rafols, Ismael, 2011. "Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations," Journal of Informetrics, Elsevier, vol. 5(1), pages 87-100.
  12. Liv Langfeldt, 2006. "The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments," Research Evaluation, Oxford University Press, vol. 15(1), pages 31-41, April.
  13. Claire Donovan, 2007. "Introduction: Future pathways for science policy and research assessment: Metrics vs peer review, quality vs impact," Science and Public Policy, Oxford University Press, vol. 34(8), pages 538-542, October.
  14. Ben Martin & Paul Nightingale & Alfredo Yegros-Yegros, 2011. "Science and Technology Studies: Exploring the Knowledge Base," Working Papers on Innovation Studies 20111004, Centre for Technology, Innovation and Culture, University of Oslo.
  15. J Sylvan Katz, 2000. "Scale-independent indicators and research evaluation," Science and Public Policy, Oxford University Press, vol. 27(1), pages 23-36, February.
  16. deS. Price, Derek, 1984. "The science/technology relationship, the craft of experimental science, and policy for the improvement of high technology innovation," Research Policy, Elsevier, vol. 13(1), pages 3-20, February.
  17. Rhoten, Diana & Pfirman, Stephanie, 2007. "Women in interdisciplinary science: Exploring preferences and consequences," Research Policy, Elsevier, vol. 36(1), pages 56-75, February.
  18. Alan L Porter & J David Roessner & Alex S Cohen & Marty Perreault, 2006. "Interdisciplinary research: meaning, metrics and nurture," Research Evaluation, Oxford University Press, vol. 15(3), pages 187-195, December.
  19. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.
  20. Bhupatiraju, Samyukta & Nomaler, Önder & Triulzi, Giorgio & Verspagen, Bart, 2012. "Knowledge flows – Analyzing the core literature of innovation, entrepreneurship and science and technology studies," Research Policy, Elsevier, vol. 41(7), pages 1205-1218.
  21. Philip Lowe & Jeremy Phillipson, 2006. "Reflexive Interdisciplinary Research: The Making of a Research Programme on the Rural Economy and Land Use," Journal of Agricultural Economics, Wiley Blackwell, vol. 57(2), pages 165-184, 07.
  22. Cummings, Jonathon N. & Kiesler, Sara, 2007. "Coordination costs and project outcomes in multi-university collaborations," Research Policy, Elsevier, vol. 36(10), pages 1620-1634, December.
  23. Grit Laudel & Gloria Origgi, 2006. "Introduction to a special issue on the assessment of interdisciplinary research," Research Evaluation, Oxford University Press, vol. 15(1), pages 2-4, April.
  24. Nicolas CARAYOL & Thuc Uyen NGUYEN THI, 2004. "Why do Academic Scientists Engage in Interdisciplinary Research ?," Working Papers of BETA 2004-17, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
  25. Martin, Ben R. & Irvine, John, 1983. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 12(2), pages 61-90, April.
  26. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
  27. Veronica Boix Mansilla, 2006. "Assessing expert interdisciplinary work at the frontier: an empirical exploration," Research Evaluation, Oxford University Press, vol. 15(1), pages 17-29, April.
  28. Heinze, Thomas & Shapira, Philip & Rogers, Juan D. & Senker, Jacqueline M., 2009. "Organizational and institutional influences on creativity in scientific research," Research Policy, Elsevier, vol. 38(4), pages 610-623, May.
  29. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism at Risk?," American Journal of Economics and Sociology, Wiley Blackwell, vol. 69(5), pages 1495-1529, November.
  30. Frederic S. Lee, 2007. "The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities," Cambridge Journal of Economics, Oxford University Press, vol. 31(2), pages 309-325, March.
  31. Rinia, E. J. & van Leeuwen, Th. N. & van Vuren, H. G. & van Raan, A. F. J., 2001. "Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research," Research Policy, Elsevier, vol. 30(3), pages 357-361, March.
  32. David Roessner, 2000. "Quantitative and qualitative methods and measures in the evaluation of research," Research Evaluation, Oxford University Press, vol. 9(2), pages 125-132, August.
  33. Paul Nightingale & Alister Scott, 2007. "Peer review and the relevance gap: Ten suggestions for policy-makers," Science and Public Policy, Oxford University Press, vol. 34(8), pages 543-553, October.
  34. Philip Lowe & Jeremy Phillipson, 2009. "Barriers to research collaboration across disciplines: scientific paradigms and institutional practices," Environment and Planning A, Pion Ltd, London, vol. 41(5), pages 1171-1184, May.
  35. Katz, J. Sylvan & Martin, Ben R., 1997. "What is research collaboration?," Research Policy, Elsevier, vol. 26(1), pages 1-18, March.
  36. Scott E. Page, 2007. "Prologue to The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies
    [The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and So
    ," Introductory Chapters, Princeton University Press.
  37. Andy Stirling, 2007. "A General Framework for Analysing Diversity in Science, Technology and Society," SPRU Working Paper Series 156, SPRU - Science and Technology Policy Research, University of Sussex.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Martin, Ben & Nightingale, P. & Yegros-Yegros, A., 2011. "Science and Technology Studies: Exploring the Knowledge Base," ESRC Centre for Business Research - Working Papers wp427, ESRC Centre for Business Research.
  2. Clausen, Tommy & Fagerberg, Jan & Gulbrandsen, Magnus, 2012. "Mobilizing for change: A study of research units in emerging scientific fields," Research Policy, Elsevier, vol. 41(7), pages 1249-1261.
  3. Brooks, Chris & Fenton, Evelyn M. & Walker, James T., 2014. "Gender and the evaluation of research," Research Policy, Elsevier, vol. 43(6), pages 990-1001.
  4. Fagerberg, Jan & Landström, Hans & Martin, Ben R., 2012. "Exploring the emerging knowledge base of ‘the knowledge society’," Research Policy, Elsevier, vol. 41(7), pages 1121-1131.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:41:y:2012:i:7:p:1262-1282. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.