IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i7d10.1007_s11192-021-04004-y.html
   My bibliography  Save this article

Everyone onboard? Participation ratios as a metric for research activity assessments within young universities

Author

Listed:
  • Frode Eika Sandnes

    (Oslo Metropolitan University
    Kristiania University College)

Abstract

Research productivity metrics designed for international and national comparison of institutions may be uncritically adopted for research assessments, especially within young universities with short research traditions. It is argued that such measures can be misleading when assessing small groups of researchers such as within a department, and that they may provide limited usefulness for leaders at different levels of an organization. Instead, the ratio of participation is proposed as a research evaluation metric for smaller groups of researchers. Publication data from Norway are used as examples. The proposed metric is simple to implement and understand, and therefore holds potential for young universities that are strategically strengthening their research capabilities.

Suggested Citation

  • Frode Eika Sandnes, 2021. "Everyone onboard? Participation ratios as a metric for research activity assessments within young universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6105-6113, July.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:7:d:10.1007_s11192-021-04004-y
    DOI: 10.1007/s11192-021-04004-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-021-04004-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-021-04004-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Frode Eika Sandnes, 2018. "Do Norwegian academics who publish more earn higher salaries?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 263-281, April.
    2. Alexis-Michel Mugabushaka & Anthi Kyriakou & Theo Papazoglou, 2016. "Bibliometric indicators of interdisciplinarity: the potential of the Leinster–Cobbold diversity indices to study disciplinary diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 593-607, May.
    3. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    4. Estela Hernández-Martín & Fernando Calle & Juan C. Dueñas & Miguel Holgado & Asunción Gómez-Pérez, 2019. "Participation of women in doctorate, research, innovation, and management activities at Universidad Politécnica de Madrid: analysis of the decade 2006–2016," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1059-1089, September.
    5. Wen Lou & Yuehua Zhao & Yuchen Chen & Jin Zhang, 2018. "Research or management? An investigation of the impact of leadership roles on the research performance of academic administrators," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 191-209, October.
    6. Frode Eika Sandnes, 2020. "A simple back-of-the-envelope test for self-citations using Google Scholar author profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1685-1689, August.
    7. Giovanni Abramo & Ciriaco Andrea D’Angelo & Francesco Rosati, 2013. "Measuring institutional research productivity for the life sciences: the importance of accounting for the order of authors in the byline," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 779-795, December.
    8. Ugo Moschini & Elena Fenialdi & Cinzia Daraio & Giancarlo Ruocco & Elisa Molinari, 2020. "A comparison of three multidisciplinarity indices based on the diversity of Scopus subject areas of authors’ documents, their bibliography and their citing papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1145-1158, November.
    9. Tove Faber Frandsen & Mette Brandt Eriksen & David Mortan Grøne Hammer & Janne Buck Christensen, 2019. "Fragmented publishing: a large-scale study of health science," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1729-1743, June.
    10. Tobias Kiesslich & Marlena Beyreis & Georg Zimmermann & Andreas Traweger, 2021. "Citation inequality and the Journal Impact Factor: median, mean, (does it) matter?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1249-1269, February.
    11. Richard Klavans & Kevin W. Boyack, 2008. "Thought leadership: A new indicator for national and institutional comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 239-250, May.
    12. Stephen A. Rhoades, 1993. "The Herfindahl-Hirschman index," Federal Reserve Bulletin, Board of Governors of the Federal Reserve System (U.S.), issue Mar, pages 188-189.
    13. Nancy K. Bayers, 2005. "Using ISI data in the analysis of German national and institutional research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 155-163, January.
    14. Cracau, Daniel & Lima, José E. Durán, 2016. "On the Normalized Herfindahl-Hirschman Index: A Technical Note," International Journal on Food System Dynamics, International Center for Management, Communication, and Research, vol. 7(4), pages 1-5, August.
    15. Richard Klavans & Kevin W. Boyack, 2010. "Toward an objective, reliable and accurate method for measuring research leadership," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 539-553, March.
    16. François Brischoux & Frédéric Angelier, 2015. "Academia’s never-ending selection for productivity," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 333-336, April.
    17. Akio Matsumoto & Ugo Merlone & Ferenc Szidarovszky, 2012. "Some notes on applying the Herfindahl--Hirschman Index," Applied Economics Letters, Taylor & Francis Journals, vol. 19(2), pages 181-184, February.
    18. Siluo Yang & Feng Ma & Yanhui Song & Junping Qiu, 2010. "A longitudinal analysis of citation distribution breadth for Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(3), pages 755-765, December.
    19. Frederik T. Verleysen & Truyken L. B. Ossenblok, 2017. "Profiles of monograph authors in the social sciences and humanities: an analysis of productivity, career stage, co-authorship, disciplinary affiliation and gender, based on a regional bibliographic da," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1673-1686, June.
    20. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.
    21. Kjetil K. Haugen & Frode E. Sandnes, 2016. "The new Norwegian incentive system for publication: from bad to worse," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1299-1306, November.
    22. Paula Leite & Rogério Mugnaini & Jacqueline Leta, 2011. "A new indicator for international visibility: exploring Brazilian scientific community," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 311-319, July.
    23. Domingo Docampo, 2011. "Erratum to: On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 237-237, January.
    24. Domingo Docampo & Lawrence Cram, 2017. "Academic performance and institutional resources: a cross-country analysis of research universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 739-764, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Evelyn Eika & Frode Eika Sandnes, 2022. "Starstruck by journal prestige and citation counts? On students’ bias and perceptions of trustworthiness according to clues in publication references," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6363-6390, November.
    2. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    3. Frode Eika Sandnes, 2021. "A bibliometric study of human–computer interaction research activity in the Nordic-Baltic Eight countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4733-4767, June.
    4. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    5. Gnewuch, Matthias & Wohlrabe, Klaus, 2018. "Super-efficiency of education institutions: an application to economics departments," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 26, pages 610-623.
    6. Chaocheng He & Jiang Wu & Qingpeng Zhang, 2021. "Characterizing research leadership on geographically weighted collaboration network," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4005-4037, May.
    7. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    8. Seiler, Christian & Wohlrabe, Klaus, 2013. "Archetypal scientists," Journal of Informetrics, Elsevier, vol. 7(2), pages 345-356.
    9. Ricardo Arencibia-Jorge & Rosa Lidia Vega-Almeida & José Luis Jiménez-Andrade & Humberto Carrillo-Calvet, 2022. "Evolutionary stages and multidisciplinary nature of artificial intelligence research," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5139-5158, September.
    10. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    11. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    12. Seiler, Christian & Wohlrabe, Klaus, 2012. "Ranking economists on the basis of many indicators: An alternative approach using RePEc data," Journal of Informetrics, Elsevier, vol. 6(3), pages 389-402.
    13. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    14. Teodoro Luque-Martínez & Salvador Barrio-García, 2016. "Constructing a synthetic indicator of research activity," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1049-1064, September.
    15. Osmo Kivinen & Juha Hedman & Kalle Artukka, 2017. "Scientific publishing and global university rankings. How well are top publishing universities recognized?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 679-695, July.
    16. Frode Eika Sandnes, 2020. "A simple back-of-the-envelope test for self-citations using Google Scholar author profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1685-1689, August.
    17. Olcay, Gokcen Arkali & Bulu, Melih, 2017. "Is measuring the knowledge creation of universities possible?: A review of university rankings," Technological Forecasting and Social Change, Elsevier, vol. 123(C), pages 153-160.
    18. Chiang Kao & Shiang-Tai Liu & Hwei-Lan Pao, 2012. "Assessing improvement in management research in Taiwan," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 75-87, July.
    19. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    20. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.

    More about this item

    Keywords

    Research assessment; Aggregation; Research productivity; Higher education leadership; Herfindahl–Hirschman index; Simpson diversity index;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:7:d:10.1007_s11192-021-04004-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.