The lack of meaningful boundary differences between journal impact factor quartiles undermines their independent use in research evaluation
Author
Abstract
Suggested Citation
DOI: 10.1007/s11192-020-03801-1
Download full text from publisher
As the access to this document is restricted, you may want to search for a different version of it.
References listed on IDEAS
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2018. "Double rank analysis for research assessment," Journal of Informetrics, Elsevier, vol. 12(1), pages 31-41.
- Juan Miguel Campanario, 2014. "The effect of citations on the significance of decimal places in the computation of journal impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 289-298, May.
- Wang, Qi & Waltman, Ludo, 2016. "Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 10(2), pages 347-364.
- Ruth Müller & Sarah de Rijcke, 2017. "Exploring the epistemic impacts of academic performance indicators in the life sciences," Research Evaluation, Oxford University Press, vol. 26(3), pages 157-168.
- Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011.
"The skewness of science in 219 sub-fields and a number of aggregates,"
Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
- Ruiz-Castillo, Javier & Ortuño-Ortin, Ignacio & Albarran, Pedro & Crespo, Juan A., 2010. "The Skewness of Science in 219 Sub-Fields and a Number of Aggregates," CEPR Discussion Papers 8126, C.E.P.R. Discussion Papers.
- Albarrán, Pedro & Crespo, Juan A. & Ortuño, Ignacio, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," UC3M Working papers. Economics we1109, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Crespo, Juan A. & Albarrán, Pedro & Ortuño, Ignacio, 2010. "The skewness of science in 219 sub-fields and a number of aggregates," UC3M Working papers. Economics we1038, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Benjamin M. Althouse & Jevin D. West & Carl T. Bergstrom & Theodore Bergstrom, 2009.
"Differences in impact factor across fields and over time,"
Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 27-34, January.
- Althouse, Benjamin M. & West, Jevin D. & Bergstrom, Ted C & Bergstrom, Carl T., 2008. "Differences in Impact Factor Across Fields and Over Time," University of California at Santa Barbara, Economics Working Paper Series qt76h442pg, Department of Economics, UC Santa Barbara.
- Loet Leydesdorff & Lutz Bornmann & Jonathan Adams, 2019. "The integrated impact indicator revisited (I3*): a non-parametric alternative to the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1669-1694, June.
- George A. Lozano & Vincent Larivière & Yves Gingras, 2012. "The weakening relationship between the impact factor and papers' citations in the digital age," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(11), pages 2140-2145, November.
- Fei Shu & Wei Quan & Bikun Chen & Junping Qiu & Cassidy R. Sugimoto & Vincent Larivière, 2020. "The role of Web of Science publications in China’s tenure system," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1683-1695, March.
- David I. Stern, 2013.
"Uncertainty Measures for Economics Journal Impact Factors,"
Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
- David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Crawford School Research Papers 1302, Crawford School of Public Policy, The Australian National University.
- Ruiz-Castillo, Javier & Costas, Rodrigo, 2018.
"Individual and field citation distributions in 29 broad scientific fields,"
Journal of Informetrics, Elsevier, vol. 12(3), pages 868-892.
- Costas, Rodrigo, 2018. "Individual and Field Citation Distributions in 29 Broad Scientific Fields," UC3M Working papers. Economics 26100, Universidad Carlos III de Madrid. Departamento de EconomÃa.
- Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
- Éric Archambault & Vincent Larivière, 2009. "History of the journal impact factor: Contingencies and consequences," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 635-649, June.
- Pajić, Dejan, 2015. "On the stability of citation-based journal rankings," Journal of Informetrics, Elsevier, vol. 9(4), pages 990-1006.
- Loet Leydesdorff & Paul Wouters & Lutz Bornmann, 2016. "Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2129-2150, December.
- Ruth Müller & Sarah de Rijcke, 2017. "Thinking with Indicators. Exploring the Epistemic Impacts of Academic Performance Indicators in the Life Sciences," Research Evaluation, Oxford University Press, vol. 26(4), pages 361-361.
- Loet Leydesdorff & Lutz Bornmann, 2016. "The operationalization of “fields” as WoS subject categories (WCs) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies”," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(3), pages 707-714, March.
- Ruben Miranda & Esther Garcia-Carpintero, 2019. "Comparison of the share of documents and citations from different quartile journals in 25 research areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 479-501, October.
- A. I. Pudovkin & Eugene Garfield, 2012. "Rank normalization of impact factors will resolve Vanclay’s dilemma with TRIF," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 409-412, August.
- George A. Lozano & Vincent Larivière & Yves Gingras, 2012. "The weakening relationship between the impact factor and papers' citations in the digital age," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(11), pages 2140-2145, November.
- J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & J. Martinez-Baena, 2012. "On first quartile journals which are not of highest impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 925-943, March.
- Brito, Ricardo & Rodríguez-Navarro, Alonso, 2019. "Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?," Journal of Informetrics, Elsevier, vol. 13(1), pages 314-324.
- Jerome K. Vanclay, 2009. "Bias in the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 3-12, January.
- Weishu Liu & Guangyuan Hu & Mengdi Gu, 2016. "The probability of publishing in first-quartile journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1273-1276, March.
- Dag W. Aksnes, 2003. "A macro study of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 235-246, February.
- Thelwall, Mike, 2016. "Are the discretised lognormal and hooked power law distributions plausible for citation data?," Journal of Informetrics, Elsevier, vol. 10(2), pages 454-470.
- Jerome K. Vanclay, 2012. "Impact factor: outdated artefact or stepping-stone to journal certification?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 211-238, August.
- Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
- Lutz Bornmann, 2017. "Confidence intervals for Journal Impact Factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1869-1871, June.
- Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Gabriel-Alexandru Vîiu & Mihai Păunescu, 2021. "The citation impact of articles from which authors gained monetary rewards based on journal metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4941-4974, June.
- Ruben Miranda & Esther Garcia-Carpintero, 2019. "Comparison of the share of documents and citations from different quartile journals in 25 research areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 479-501, October.
- Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
- Zsolt Kohus & Márton Demeter & László Kun & Eszter Lukács & Katalin Czakó & Gyula Péter Szigeti, 2022. "A Study of the Relation between Byline Positions of Affiliated/Non-Affiliated Authors and the Scientific Impact of European Universities in Times Higher Education World University Rankings," Sustainability, MDPI, vol. 14(20), pages 1-14, October.
- Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
- Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
- Alonso Rodríguez-Navarro & Ricardo Brito, 2019. "Probability and expected frequency of breakthroughs: basis and use of a robust method of research assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 213-235, April.
- Juan Miguel Campanario, 2018. "Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 111-130, April.
- Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Costa, 2023. "Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1877-1894, March.
- Brito, Ricardo & Rodríguez-Navarro, Alonso, 2019. "Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?," Journal of Informetrics, Elsevier, vol. 13(1), pages 314-324.
- Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
- Pech, Gerson & Delgado, Catarina, 2021. "Screening the most highly cited papers in longitudinal bibliometric studies and systematic literature reviews of a research field or journal: Widespread used metrics vs a percentile citation-based app," Journal of Informetrics, Elsevier, vol. 15(3).
- David I Stern, 2014.
"High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information,"
PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
- David I. Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," Crawford School Research Papers 1406, Crawford School of Public Policy, The Australian National University.
- Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
- Rodríguez-Navarro, Alonso & Brito, Ricardo, 2024. "Rank analysis of most cited publications, a new approach for research assessments," Journal of Informetrics, Elsevier, vol. 18(2).
- Yves Fassin, 2021. "Does the Financial Times FT50 journal list select the best management and economics journals?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5911-5943, July.
- Michael McAleer & Judit Olah & Jozsef Popp, 2018.
"Pros and Cons of the Impact Factor in a Rapidly Changing Digital World,"
Tinbergen Institute Discussion Papers
18-014/III, Tinbergen Institute.
- Michael McAleer & Judit Oláh & József Popp, 2018. "Pros and cons of the impact factor in a rapidly changing digital world," Documentos de Trabajo del ICAE 2018-06, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico.
- McAleer, M.J. & Oláh, J. & Popp, J., 2018. "Pros and Cons of the Impact Factor in a Rapidly Changing Digital World," Econometric Institute Research Papers EI2018-11, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
- Lutz Bornmann & Klaus Wohlrabe, 2019.
"Normalisation of citation impact in economics,"
Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
- Wohlrabe, Klaus & Bornmann, Lutz, 2017. "Normalization of citation impact in economics," MPRA Paper 80384, University Library of Munich, Germany.
- Lutz Bornmann & Klaus Wohlrabe, 2017. "Normalization of Citation Impact in Economics," CESifo Working Paper Series 6592, CESifo.
- de Carvalho, Gustavo Dambiski Gomes & Sokulski, Carla Cristiane & da Silva, Wesley Vieira & de Carvalho, Hélio Gomes & de Moura, Rafael Vignoli & de Francisco, Antonio Carlos & da Veiga, Claudimar Per, 2020. "Bibliometrics and systematic reviews: A comparison between the Proknow-C and the Methodi Ordinatio," Journal of Informetrics, Elsevier, vol. 14(3).
- Bornmann, Lutz & Williams, Richard, 2017. "Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResearcherID data," Journal of Informetrics, Elsevier, vol. 11(3), pages 788-799.
More about this item
Keywords
Journal impact factor (JIF); JIF quartiles; Journal citation reports (JCR); JCR subject categories; Meaningful differences; Local quartile similarity;All these keywords.
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:2:d:10.1007_s11192-020-03801-1. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.