IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v82y2010i3d10.1007_s11192-010-0188-6.html
   My bibliography  Save this article

Toward an objective, reliable and accurate method for measuring research leadership

Author

Listed:
  • Richard Klavans

    (SciTech Strategies, Inc.)

  • Kevin W. Boyack

    (SciTech Strategies, Inc.)

Abstract

We compare a new method for measuring research leadership with the traditional method. Both methods are objective and reliable, utilize standard citation databases, and are easily replicated. The traditional method uses partitions of science based on journal categories, and has been extensively used to measure national leadership patterns in science, including those appearing in the NSF Science & Engineering Indicators Reports and in prominent journals such as Science and Nature. Our new method is based on co-citation techniques at the paper level. It was developed with the specific intent of measuring research leadership at a university, and was then extended to examine national patterns of research leadership. A comparison of these two methods provides compelling evidence that the traditional method grossly underestimates research leadership in most countries. The new method more accurately portrays the actual patterns of research leadership at the national level.

Suggested Citation

  • Richard Klavans & Kevin W. Boyack, 2010. "Toward an objective, reliable and accurate method for measuring research leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 539-553, March.
    • Handle: RePEc:spr:scient:v:82:y:2010:i:3:d:10.1007_s11192-010-0188-6
    DOI: 10.1007/s11192-010-0188-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-010-0188-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-010-0188-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kevin W. Boyack, 2009. "Using detailed maps of science to identify potential collaborations," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(1), pages 27-44, April.
    2. Loet Leydesdorff & Caroline Wagner, 2009. "Is the United States losing ground in science? A global perspective on the world science system," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 23-36, January.
    3. Wolfgang Glänzel & Bart Thijs & András Schubert & Koenraad Debackere, 2009. "Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 165-188, January.
    4. E. Bassecoulard & M. Zitt, 1999. "Indicators in a research institute: A multi-level classification of scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 44(3), pages 323-345, March.
    5. Richard Klavans & Kevin W. Boyack, 2008. "Thought leadership: A new indicator for national and institutional comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 239-250, May.
    6. Richard Klavans & Kevin W. Boyack, 2006. "Quantitative evaluation of large maps of science," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 475-499, September.
    7. Henry Small, 1999. "Visualizing science by citation mapping," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(9), pages 799-813.
    8. Wolfgang Glänzel & András Schubert, 2003. "A new classification scheme of science fields and subfields designed for scientometric evaluation purposes," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(3), pages 357-367, March.
    9. David A. King, 2004. "The scientific impact of nations," Nature, Nature, vol. 430(6997), pages 311-316, July.
    10. Clara Calero-Medina & Carmen López-Illescas & Martijn S Visser & Henk F Moed, 2008. "Important factors when interpreting bibliometric rankings of world universities: an example from oncology," Research Evaluation, Oxford University Press, vol. 17(1), pages 71-81, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ying Huang & Wolfgang Glänzel & Lin Zhang, 2021. "Tracing the development of mapping knowledge domains," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6201-6224, July.
    2. Ludo Waltman & Nees Jan Eck, 2012. "A new methodology for constructing a publication-level classification system of science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2378-2392, December.
    3. An, Lu & Yu, Chuanming & Li, Gang, 2014. "Visual topical analysis of Chinese and American Library and Information Science research institutions," Journal of Informetrics, Elsevier, vol. 8(1), pages 217-233.
    4. Chaocheng He & Jiang Wu & Qingpeng Zhang, 2021. "Characterizing research leadership on geographically weighted collaboration network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4005-4037, May.
    5. Pranpreya Sriwannawit & Ulf Sandström, 2015. "Large-scale bibliometric review of diffusion research," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1615-1645, February.
    6. Chaocheng He & Jiang Wu & Qingpeng Zhang, 2022. "Proximity‐aware research leadership recommendation in research collaboration via deep neural networks," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(1), pages 70-89, January.
    7. Rosa Rodriguez-Sánchez & J. A. García & J. Fdez-Valdivia, 2014. "Evolutionary games between subject categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 869-888, October.
    8. Frode Eika Sandnes, 2021. "Everyone onboard? Participation ratios as a metric for research activity assessments within young universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6105-6113, July.
    9. Kevin W. Boyack, 2017. "Thesaurus-based methods for mapping contents of publication sets," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 1141-1155, May.
    10. Juan Miguel Campanario, 2018. "Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 111-130, April.
    11. Sánchez-Jiménez, Rodrigo & Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2017. "The role of guarantor in scientific collaboration: The neighbourhood matters," Journal of Informetrics, Elsevier, vol. 11(1), pages 103-116.
    12. Berndt Jesenko & Christian Schlögl, 2021. "The effect of web of science subject categories on clustering: the case of data-driven methods in business and economic sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6785-6801, August.
    13. Klavans, Richard & Boyack, Kevin W., 2014. "Mapping altruism," Journal of Informetrics, Elsevier, vol. 8(2), pages 431-447.
    14. Hisato Tashiro & Antonio Lau & Junichiro Mori & Nobuzumi Fujii & Yuya Kajikawa, 2012. "E-mail networks and leadership performance," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(3), pages 600-606, March.
    15. Juan Miguel Campanario, 2017. "JIF-Plots: using plots of citations versus citable items as a tool to study journals and subject categories and discover new scientometric relationships," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1141-1154, November.
    16. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    2. Ismael Rafols & Alan Porter & Loet Leydesdorff, 2009. "Overlay Maps of Science: a New Tool for Research Policy," SPRU Working Paper Series 179, SPRU - Science Policy Research Unit, University of Sussex Business School.
    3. Francesco Giuliani & Michele Pio De Petris & Giovanni Nico, 2010. "Assessing scientific collaboration through coauthorship and content sharing," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 13-28, October.
    4. Richard Klavans & Kevin W. Boyack, 2008. "Thought leadership: A new indicator for national and institutional comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 239-250, May.
    5. Hyeonchae Yang & Woo-Sung Jung, 2015. "A strategic management approach for Korean public research institutes based on bibliometric investigation," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(4), pages 1437-1464, July.
    6. Patrick Herron & Aashish Mehta & Cong Cao & Timothy Lenoir, 2016. "Research diversification and impact: the case of national nanoscience development," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 629-659, November.
    7. Yves Gingras & Mahdi Khelfaoui, 2018. "Assessing the effect of the United States’ “citation advantage” on other countries’ scientific impact as measured in the Web of Science (WoS) database," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 517-532, February.
    8. Silva, F.N. & Viana, M.P. & Travençolo, B.A.N. & Costa, L. da F., 2011. "Investigating relationships within and between category networks in Wikipedia," Journal of Informetrics, Elsevier, vol. 5(3), pages 431-438.
    9. Gregory J Hather & Winston Haynes & Roger Higdon & Natali Kolker & Elizabeth A Stewart & Peter Arzberger & Patrick Chain & Dawn Field & B Robert Franza & Biaoyang Lin & Folker Meyer & Vural Ozdemir & , 2010. "The United States of America and Scientific Research," PLOS ONE, Public Library of Science, vol. 5(8), pages 1-9, August.
    10. Ying Huang & Wolfgang Glänzel & Lin Zhang, 2021. "Tracing the development of mapping knowledge domains," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6201-6224, July.
    11. Jielan Ding & Per Ahlgren & Liying Yang & Ting Yue, 2018. "Disciplinary structures in Nature, Science and PNAS: journal and country levels," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1817-1852, September.
    12. Wolfgang Glänzel & András Schubert & Bart Thijs & Koenraad Debackere, 2011. "A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 415-424, May.
    13. Michel Zitt, 2015. "Meso-level retrieval: IR-bibliometrics interplay and hybrid citation-words methods in scientific fields delineation," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2223-2245, March.
    14. Boyack, Kevin W. & Klavans, Richard, 2008. "Measuring science–technology interaction using rare inventor–author names," Journal of Informetrics, Elsevier, vol. 2(3), pages 173-182.
    15. Wolfgang Glänzel & Koenraad Debackere, 2022. "Various aspects of interdisciplinarity in research and how to quantify and measure those," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5551-5569, September.
    16. Chaocheng He & Jiang Wu & Qingpeng Zhang, 2021. "Characterizing research leadership on geographically weighted collaboration network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4005-4037, May.
    17. Zaida Chinchilla-Rodríguez & Grisel Zacca-González & Benjamín Vargas-Quesada & Félix Moya-Anegón, 2016. "Benchmarking scientific performance by decomposing leadership of Cuban and Latin American institutions in Public Health," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1239-1264, March.
    18. Zhenyue Zhao & Xuelian Pan & Weina Hua, 2021. "Comparative analysis of the research productivity, publication quality, and collaboration patterns of top ranked library and information science schools in China and the United States," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 931-950, February.
    19. Yong Yi & Wei Qi & Dandan Wu, 2013. "Are CIVETS the next BRICs? A comparative analysis from scientometrics perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 615-628, February.
    20. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:82:y:2010:i:3:d:10.1007_s11192-010-0188-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.