IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v92y2012i3d10.1007_s11192-012-0632-x.html
   My bibliography  Save this article

Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds

Author

Listed:
  • Per Ahlgren

    (University Library, Stockholm University
    Inforsk, Umeå University)

  • Cristian Colliander

    (Inforsk, Umeå University)

  • Olle Persson

    (Inforsk, Umeå University)

Abstract

We compared three different bibliometric evaluation approaches: two citation-based approaches and one based on manual classification of publishing channels into quality levels. Publication data for two universities was used, and we worked with two levels of analysis: article and department. For the article level, we investigated the predictive power of field normalized citation rates and field normalized journal impact with respect to journal level. The results for the article level show that evaluation of journals based on citation impact correlate rather well with manual classification of journals into quality levels. However, the prediction from field normalized citation rates to journal level was only marginally better than random guessing. At the department level, we studied three different indicators in the context of research fund allocation within universities and the extent to which the three indicators produce different distributions of research funds. It turned out that the three distributions of relative indicator values were very similar, which in turn yields that the corresponding distributions of hypothetical research funds would be very similar.

Suggested Citation

  • Per Ahlgren & Cristian Colliander & Olle Persson, 2012. "Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 767-780, September.
    • Handle: RePEc:spr:scient:v:92:y:2012:i:3:d:10.1007_s11192-012-0632-x
    DOI: 10.1007/s11192-012-0632-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0632-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0632-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aldo Geuna, 2001. "The Changing Rationale for European University Research Funding: Are There Negative Unintended Consequences?," Journal of Economic Issues, Taylor & Francis Journals, vol. 35(3), pages 607-632, September.
    2. Per O. Seglen, 1994. "Causal relationship between article citedness and journal impact," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 45(1), pages 1-11, January.
    3. Ulf Sandström & Erik Sandström, 2009. "The field factor: towards a metric for academic institutions," Research Evaluation, Oxford University Press, vol. 18(3), pages 243-250, September.
    4. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    5. Peter Weingart, 2005. "Impact of bibliometrics upon the science system: Inadvertent consequences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 117-131, January.
    6. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    7. Opthof, Tobias & Leydesdorff, Loet, 2010. "Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance," Journal of Informetrics, Elsevier, vol. 4(3), pages 423-430.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gunnar Sivertsen, 2016. "Data integration in Scandinavia," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 849-855, February.
    2. Daniela Filippo & Rafael Aleixandre-Benavent & Elías Sanz-Casado, 2020. "Toward a classification of Spanish scholarly journals in social sciences and humanities considering their impact and visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1709-1732, November.
    3. Ahlgren, Per & Waltman, Ludo, 2014. "The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs. Norwegian quality assessments," Journal of Informetrics, Elsevier, vol. 8(4), pages 985-996.
    4. Fernandez Martinez, Roberto & Lostado Lorza, Ruben & Santos Delgado, Ana Alexandra & Piedra, Nelson, 2021. "Use of classification trees and rule-based models to optimize the funding assignment to research projects: A case study of UTPL," Journal of Informetrics, Elsevier, vol. 15(1).
    5. Saarela, Mirka & Kärkkäinen, Tommi & Lahtonen, Tommi & Rossi, Tuomo, 2016. "Expert-based versus citation-based ranking of scholarly and scientific publication channels," Journal of Informetrics, Elsevier, vol. 10(3), pages 693-718.
    6. Elías Sanz-Casado & Daniela Filippo & Rafael Aleixandre Benavent & Vidar Røeggen & Janne Pölönen, 2021. "Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 9031-9049, November.
    7. Star X. Zhao & Wen Lou & Alice M. Tan & Shuang Yu, 2018. "Do funded papers attract more usage?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 153-168, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Müller, Harry, 2012. "Die Zitationshäufigkeit als Qualitätsindikator im Rahmen der Forschungsleistungsmessung," Discussion Papers of the Institute for Organisational Economics 1/2012, University of Münster, Institute for Organisational Economics.
    3. Aksnes, Dag W. & Rip, Arie, 2009. "Researchers' perceptions of citations," Research Policy, Elsevier, vol. 38(6), pages 895-905, July.
    4. Abramo, Giovanni & Cicero, Tindaro & D’Angelo, Ciriaco Andrea, 2012. "The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems," Journal of Informetrics, Elsevier, vol. 6(2), pages 155-168.
    5. Sabrina Petersohn & Thomas Heinze, 2018. "Professionalization of bibliometric research assessment. Insights from the history of the Leiden Centre for Science and Technology Studies (CWTS)," Science and Public Policy, Oxford University Press, vol. 45(4), pages 565-578.
    6. Benedetto Lepori & Aldo Geuna & Valerio Veglio, 2017. "A Typology of European Research Universities. Differentiation, Layering and Resource Distribution," SPRU Working Paper Series 2017-01, SPRU - Science Policy Research Unit, University of Sussex Business School.
    7. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Rosati, Francesco, 2013. "The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences," Journal of Informetrics, Elsevier, vol. 7(1), pages 198-208.
    8. Larivière, Vincent & Gingras, Yves, 2011. "Averages of ratios vs. ratios of averages: An empirical analysis of four levels of aggregation," Journal of Informetrics, Elsevier, vol. 5(3), pages 392-399.
    9. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2011. "A national-scale cross-time analysis of university research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 399-413, May.
    10. Civera, Alice & Lehmann, Erik E. & Paleari, Stefano & Stockinger, Sarah A.E., 2020. "Higher education policy: Why hope for quality when rewarding quantity?," Research Policy, Elsevier, vol. 49(8).
    11. Sjoerd Hardeman, 2013. "Organization level research in scientometrics: a plea for an explicit pragmatic approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1175-1194, March.
    12. Andrea Bonaccorsi & Luca Secondi & Enza Setteducati & Alessio Ancaiani, 2014. "Participation and commitment in third-party research funding: evidence from Italian Universities," The Journal of Technology Transfer, Springer, vol. 39(2), pages 169-198, April.
    13. Migheli, Matteo & Zotti, Roberto, 2020. "The strange case of the Matthew effect and beauty contests: Research evaluation and specialisation in Italian universities," Socio-Economic Planning Sciences, Elsevier, vol. 71(C).
    14. Maaike Verbree & Edwin Horlings & Peter Groenewegen & Inge Weijden & Peter Besselaar, 2015. "Organizational factors influencing scholarly performance: a multivariate study of biomedical research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 25-49, January.
    15. Ludo Waltman & Nees Jan Eck & Thed N. Leeuwen & Martijn S. Visser & Anthony F. J. Raan, 2011. "Towards a new crown indicator: an empirical analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 467-481, June.
    16. Lutz Bornmann & Alexander Tekles & Loet Leydesdorff, 2019. "How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1187-1205, May.
    17. Benedetto, Lepori & Geuna, Aldo & Veglio, Valerio, 2017. "A Typology of European Universities. Differentiation and resource distribution," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201705, University of Turin.
    18. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    19. Zhou, Ping & Zhong, Yongfeng, 2012. "The citation-based indicator and combined impact indicator—New options for measuring impact," Journal of Informetrics, Elsevier, vol. 6(4), pages 631-638.
    20. Rons, Nadine, 2012. "Partition-based Field Normalization: An approach to highly specialized publication records," Journal of Informetrics, Elsevier, vol. 6(1), pages 1-10.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:92:y:2012:i:3:d:10.1007_s11192-012-0632-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.