IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v73y2022i12p1741-1754.html
   My bibliography  Save this article

Uses of the Journal Impact Factor in national journal rankings in China and Europe

Author

Listed:
  • Emanuel Kulczycki
  • Ying Huang
  • Alesia A. Zuccala
  • Tim C. E. Engels
  • Antonio Ferrara
  • Raf Guns
  • Janne Pölönen
  • Gunnar Sivertsen
  • Zehra Taşkın
  • Lin Zhang

Abstract

This paper investigates different uses of the Journal Impact Factor (JIF) in national journal rankings and discusses the merits of supplementing metrics with expert assessment. Our focus is national journal rankings used as evidence to support decisions about the distribution of institutional funding or career advancement. The seven countries under comparison are China, Denmark, Finland, Italy, Norway, Poland, and Turkey—and the region of Flanders in Belgium. With the exception of Italy, top‐tier journals used in national rankings include those classified at the highest level, or according to tier, or points implemented. A total of 3,565 (75.8%) out of 4,701 unique top‐tier journals were identified as having a JIF, with 55.7% belonging to the first Journal Impact Factor quartile. Journal rankings in China, Flanders, Poland, and Turkey classify journals with a JIF as being top‐tier, but only when they are in the first quartile of the Average Journal Impact Factor Percentile. Journal rankings that result from expert assessment in Denmark, Finland, and Norway regularly classify journals as top‐tier outside the first quartile, particularly in the social sciences and humanities. We conclude that experts, when tasked with metric‐informed journal rankings, take into account quality dimensions that are not covered by JIFs.

Suggested Citation

  • Emanuel Kulczycki & Ying Huang & Alesia A. Zuccala & Tim C. E. Engels & Antonio Ferrara & Raf Guns & Janne Pölönen & Gunnar Sivertsen & Zehra Taşkın & Lin Zhang, 2022. "Uses of the Journal Impact Factor in national journal rankings in China and Europe," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(12), pages 1741-1754, December.
    • Handle: RePEc:bla:jinfst:v:73:y:2022:i:12:p:1741-1754
    DOI: 10.1002/asi.24706
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24706
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24706?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Emanuel Kulczycki, 2019. "Field patterns of scientometric indicators use for presenting research portfolio for assessment," Research Evaluation, Oxford University Press, vol. 28(2), pages 169-181.
    2. Kaare Aagaard & Carter Bloch & Jesper W. Schneider, 2015. "Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator," Research Evaluation, Oxford University Press, vol. 24(2), pages 106-117.
    3. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    4. Przemysław Korytkowski & Emanuel Kulczycki, 2019. "Examining how country-level science policy shapes publication patterns: the case of Poland," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1519-1543, June.
    5. Serhii Nazarovets, 2020. "Controversial practice of rewarding for publications in national journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 813-818, July.
    6. Diana Hicks & Paul Wouters & Ludo Waltman & Sarah de Rijcke & Ismael Rafols, 2015. "Bibliometrics: The Leiden Manifesto for research metrics," Nature, Nature, vol. 520(7548), pages 429-431, April.
    7. Emanuel Kulczycki & Ewa A. Rozkosz, 2017. "Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 417-442, April.
    8. Antonio Ferrara & Andrea Bonaccorsi, 2016. "How robust is journal rating in Humanities and Social Sciences? Evidence from a large-scale, multi-method exercise," Research Evaluation, Oxford University Press, vol. 25(3), pages 279-291.
    9. Saarela, Mirka & Kärkkäinen, Tommi & Lahtonen, Tommi & Rossi, Tuomo, 2016. "Expert-based versus citation-based ranking of scholarly and scientific publication channels," Journal of Informetrics, Elsevier, vol. 10(3), pages 693-718.
    10. Paul Wouters & Cassidy R. Sugimoto & Vincent Larivière & Marie E. McVeigh & Bernd Pulverer & Sarah de Rijcke & Ludo Waltman, 2019. "Rethinking impact factors: better ways to judge a journal," Nature, Nature, vol. 569(7758), pages 621-623, May.
    11. Lin Zhang & Ronald Rousseau & Gunnar Sivertsen, 2017. "Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen's work on journal impact and research evaluation," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-18, March.
    12. Saarela, Mirka & Kärkkäinen, Tommi, 2020. "Can we automate expert-based journal rankings? Analysis of the Finnish publication indicator," Journal of Informetrics, Elsevier, vol. 14(2).
    13. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Elena Veretennik & Maria Yudkevich, 2023. "Inconsistent quality signals: evidence from the regional journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3675-3701, June.
    2. Yahui Liu & Liying Yang & Jiandong Zhang & Zhesi Shen, 2025. "Addressing the challenges in journal evaluation during the “covidization” of scientific research era : insights from the CAS journal ranking," Palgrave Communications, Palgrave Macmillan, vol. 12(1), pages 1-11, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kulczycki, Emanuel & Korzeń, Marcin & Korytkowski, Przemysław, 2017. "Toward an excellence-based research funding system: Evidence from Poland," Journal of Informetrics, Elsevier, vol. 11(1), pages 282-298.
    2. Giovanni Abramo & Ciriaco Andrea D’Angelo & Myroslava Hladchenko, 2023. "Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4589-4609, August.
    3. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    4. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    5. Mona Farouk Ali, 2025. "Investigating shifts in publication patterns after launching scientometric evaluation at Egyptian universities: an analysis of submitted research for promotion," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1751-1787, March.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2023. "Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1877-1894, March.
    7. Mona Farouk Ali, 2024. "Is there a “difference-in-difference”? The impact of scientometric evaluation on the evolution of international publications in Egyptian universities and research centres," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 1119-1154, February.
    8. Nicky Agate & Rebecca Kennison & Stacy Konkiel & Christopher P. Long & Jason Rhody & Simone Sacchi & Penelope Weber, 2020. "The transformative power of values-enacted scholarship," Palgrave Communications, Palgrave Macmillan, vol. 7(1), pages 1-12, December.
    9. Sandra Rousseau & Ronald Rousseau, 2021. "Bibliometric Techniques And Their Use In Business And Economics Research," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1428-1451, December.
    10. Csomós, György, 2020. "Introducing recalibrated academic performance indicators in the evaluation of individuals’ research performance: A case study from Eastern Europe," Journal of Informetrics, Elsevier, vol. 14(4).
    11. Hladchenko, Myroslava & Moed, Henk F., 2021. "The effect of publication traditions and requirements in research assessment and funding policies upon the use of national journals in 28 post-socialist countries," Journal of Informetrics, Elsevier, vol. 15(4).
    12. Renata Kudaibergenova & Sandugash Uzakbay & Asselya Makanova & Kymbat Ramadinkyzy & Erlan Kistaubayev & Ruslan Dussekeev & Kadyrzhan Smagulov, 2022. "Managing publication change at Al-Farabi Kazakh National University: a case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 453-479, January.
    13. Loet Leydesdorff & Paul Wouters & Lutz Bornmann, 2016. "Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2129-2150, December.
    14. Brooks, Chris & Schopohl, Lisa & Walker, James T., 2023. "Comparing perceptions of the impact of journal rankings between fields," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 90(C).
    15. Francesco Giovanni Avallone & Alberto Quagli & Paola Ramassa, 2022. "Interdisciplinary research by accounting scholars: An exploratory study," FINANCIAL REPORTING, FrancoAngeli Editore, vol. 2022(2), pages 5-34.
    16. Schmal, W. Benedikt & Haucap, Justus & Knoke, Leon, 2023. "The role of gender and coauthors in academic publication behavior," Research Policy, Elsevier, vol. 52(10).
    17. repec:osf:socarx:3kf9d_v1 is not listed on IDEAS
    18. Confraria, Hugo & Mira Godinho, Manuel & Wang, Lili, 2017. "Determinants of citation impact: A comparative analysis of the Global South versus the Global North," Research Policy, Elsevier, vol. 46(1), pages 265-279.
    19. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    20. Drivas, Kyriakos & Kremmydas, Dimitris, 2020. "The Matthew effect of a journal's ranking," Research Policy, Elsevier, vol. 49(4).
    21. Mariani, Manuel Sebastian & Medo, Matúš & Zhang, Yi-Cheng, 2016. "Identification of milestone papers through time-balanced network centrality," Journal of Informetrics, Elsevier, vol. 10(4), pages 1207-1223.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:73:y:2022:i:12:p:1741-1754. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.