IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i1d10.1007_s11192-020-03764-3.html
   My bibliography  Save this article

Number of references: a large-scale study of interval ratios

Author

Listed:
  • Jeppe Nicolaisen

    (University of Copenhagen)

  • Tove Faber Frandsen

    (University of Southern Denmark)

Abstract

The paper presents a large-scale study (covering 26,998,764 items) of the development in the number of references over time (1996–2019) in three document types (articles; reviews; notes) from seven fields (Arts and Humanities; Social Sciences; Computer Science; Mathematics; Engineering; Medicine; Physics and Astronomy). Using interval ratios instead of average numbers, the paper makes it possible to follow the development, and to locate the main causes of growth in the number of references over time. The results show significant differences between fields and document types. The number of references in journal articles and reviews are growing in all fields (except for the reviews in Arts and Humanities that remain stable over time), but at different pace; The number of references in notes are growing in some fields (again at different pace) and are stable in others. The observed growth is primarily caused by a drop in short reference lists and a corresponding increase in a bit longer and medium size reference lists. Long and very long reference lists remain much more stable in shares over time, and does therefore not contribute much to the observed growth. The results underline the importance of normalizing citation data, and for taking citation inflation into account when conducting citation analyses expanding different fields, document types, and longer time-periods.

Suggested Citation

  • Jeppe Nicolaisen & Tove Faber Frandsen, 2021. "Number of references: a large-scale study of interval ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 259-285, January.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03764-3
    DOI: 10.1007/s11192-020-03764-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03764-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03764-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Zitt, Michel, 2010. "Citing-side normalization of journal impact: A robust variant of the Audience Factor," Journal of Informetrics, Elsevier, vol. 4(3), pages 392-406.
    2. Günter Krampen, 2010. "Acceleration of citing behavior after the millennium? Exemplary bibliometric reference analyses for psychology journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 507-513, May.
    3. Helmut A. Abt & Eugene Garfield, 2002. "Is the relationship between numbers of references and paper lengths the same for all sciences?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(13), pages 1106-1112, November.
    4. John Mingers & Martin Meyer, 2017. "Normalizing Google Scholar data for use in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 1111-1121, August.
    5. Iñaki Ucar & Felipe López-Fernandino & Pablo Rodriguez-Ulibarri & Laura Sesma-Sanchez & Veronica Urrea-Micó & Joaquín Sevilla, 2014. "Growth in the number of references in engineering journal papers during the 1972–2013 period," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1855-1864, March.
    6. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    7. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    8. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    9. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    10. Petersen, Alexander M. & Pan, Raj K. & Pammolli, Fabio & Fortunato, Santo, 2019. "Methods to account for citation inflation in research evaluation," Research Policy, Elsevier, vol. 48(7), pages 1855-1865.
    11. Erwin Krauskopf, 2019. "Missing documents in Scopus: the case of the journal Enfermeria Nefrologica," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 543-547, April.
    12. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 2.
    13. Pan, Raj K. & Petersen, Alexander M. & Pammolli, Fabio & Fortunato, Santo, 2018. "The memory of science: Inflation, myopia, and the knowledge network," Journal of Informetrics, Elsevier, vol. 12(3), pages 656-678.
    14. Ben‐Ami Lipetz, 1999. "Aspects of JASIS authorship through five decades," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(11), pages 994-1003.
    15. Bornmann, Lutz & Marx, Werner, 2018. "Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics," Journal of Informetrics, Elsevier, vol. 12(3), pages 598-604.
    16. John Mingers & Martin Meyer, 2017. "Erratum to: Normalizing Google Scholar data for use in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 1123-1124, August.
    17. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
    18. Sánchez-Gil, Susana & Gorraiz, Juan & Melero-Fuentes, David, 2018. "Reference density trends in the major disciplines," Journal of Informetrics, Elsevier, vol. 12(1), pages 42-58.
    19. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    20. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    21. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Charlie Karlsson & Björn Hammarfelt, 2025. "Correction: The growth and development of Nordic regional science research 1982–2022: bibliometric evidence from thirteen regional science journals," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 74(2), pages 1-1, June.
    2. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2023. "Uncited papers in the structure of scientific communication," Journal of Informetrics, Elsevier, vol. 17(2).
    3. Petr Praus, 2024. "Analysis of journal rankings confirms that more cited articles contain more references," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 7153-7160, November.
    4. Tove Faber Frandsen & Jeppe Nicolaisen, 2023. "Defining the unscholarly publication: a bibliometric study of uncited and barely cited publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1337-1350, February.
    5. Shing-Yun Jung & Ting-Han Lin & Chia-Hung Liao & Shyan-Ming Yuan & Chuen-Tsai Sun, 2022. "Intent-Controllable Citation Text Generation," Mathematics, MDPI, vol. 10(10), pages 1-17, May.
    6. Jonathan Adams & Martin Szomszor, 2024. "National research impact is driven by global collaboration, not rising performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(5), pages 2883-2896, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    2. Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
    3. Maura Mezzetti & Ilia Negri, 2024. "Hierarchical Bayesian model to estimate and compare research productivity of Italian academic statisticians," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7443-7474, December.
    4. repec:ejw:journl:v:5:y:2008:i:3:p:294-315 is not listed on IDEAS
    5. repec:ejw:journl:v:6:y:2009:i:1:p:73-112 is not listed on IDEAS
    6. repec:ejw:journl:v:5:y:2008:i:2:p:193-198 is not listed on IDEAS
    7. Peter Hennecke, 2021. "The ECB’s New Monetary Policy Strategy," Intereconomics: Review of European Economic Policy, Springer;ZBW - Leibniz Information Centre for Economics;Centre for European Policy Studies (CEPS), vol. 56(5), pages 295-298, September.
    8. Toma Lankauskiene, 2021. "Labour Productivity Growth Determinants in the Manufacturing Sector in the Baltic States," ConScienS Conference Proceedings 025tl, Research Association for Interdisciplinary Studies.
    9. Lars P Feld & Volker Wieland, 2021. "The German Federal Constitutional Court Ruling and the European Central Bank’s Strategy," Journal of Financial Regulation, Oxford University Press, vol. 7(2), pages 217-253.
    10. Núñez Ferrer, Jorge, 2021. "Avoiding the Main Risks in the Recovery Plans of Member States," CEPS Papers 32463, Centre for European Policy Studies.
    11. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    12. repec:ejw:journl:v:5:y:2008:i:2:p:227-239 is not listed on IDEAS
    13. repec:ejw:journl:v:7:y:2010:i:1:p:4-52 is not listed on IDEAS
    14. repec:ejw:journl:v:6:y:2009:i:1:p:35-59 is not listed on IDEAS
    15. repec:ejw:journl:v:6:y:2009:i:2:p:181-194 is not listed on IDEAS
    16. Kjerstin Tevik & Geir Selbæk & Knut Engedal & Arnfinn Seim & Steinar Krokstad & Anne-S Helvik, 2019. "Mortality in older adults with frequent alcohol consumption and use of drugs with addiction potential – The Nord Trøndelag Health Study 2006-2008 (HUNT3), Norway, a population-based study," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-24, April.
    17. repec:gig:chaktu:v:40:y:2011:i:1:p:3-17 is not listed on IDEAS
    18. Yunhee Park & Hyun-Jung Yun, 2020. "A Multilevel Investigation of Fall Prevention Behavior Among Nursing Staff of South Korean Geriatric Hospitals," Global Journal of Health Science, Canadian Center of Science and Education, vol. 12(10), pages 1-97, September.
    19. repec:ejw:journl:v:4:y:2007:i:3:p:338-344 is not listed on IDEAS
    20. repec:ejw:journl:v:5:y:2008:i:1:p:59-65 is not listed on IDEAS
    21. repec:ejw:journl:v:5:y:2008:i:2:p:240-268 is not listed on IDEAS
    22. Junghyun Yoon & Dae-su Kim, 2016. "Empirical Relationships among Technological Characteristics, Global Orientation, and Internationalisation of South Korean New Ventures," Sustainability, MDPI, vol. 8(12), pages 1-13, December.
    23. repec:ejw:journl:v:5:y:2008:i:1:p:78-90 is not listed on IDEAS
    24. repec:ejw:journl:v:6:y:2009:i:2:p:218-238 is not listed on IDEAS
    25. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    26. Achim Truger, 2021. "Reform der EU-Fiskalregeln nach Corona wichtiger denn je," Wirtschaftsdienst, Springer;ZBW - Leibniz Information Centre for Economics, vol. 101(2), pages 94-98, February.
    27. Heng Zhang & Chengyou Wang & Xiao Zhou, 2017. "A Robust Image Watermarking Scheme Based on SVD in the Spatial Domain," Future Internet, MDPI, vol. 9(3), pages 1-16, August.
    28. Hyun Min Oh & Sam Bock Park & Hee Young Ma, 2020. "Corporate Sustainability Management, Earnings Transparency, and Chaebols," Sustainability, MDPI, vol. 12(10), pages 1-19, May.
    29. Vellingiri Vadivel & Pemaiah Brindha, 2017. "Wound Healing Potential of Ipomoea carnea Jacq.: An Un-Explored Herb Used in Indian Traditional System of Medicine," Global Journal of Pharmacy & Pharmaceutical Sciences, Juniper Publishers Inc., vol. 3(1), pages 1-5, June.
    30. Aldo Alvarez-Risco & Sabina Mlodzianowska & Verónica García-Ibarra & Marc A. Rosen & Shyla Del-Aguila-Arcentales, 2021. "Factors Affecting Green Entrepreneurship Intentions in Business University Students in COVID-19 Pandemic Times: Case of Ecuador," Sustainability, MDPI, vol. 13(11), pages 1-16, June.
    31. Yingyos Leechaianan & Dennis R. Longmire, 2013. "The Use of the Death Penalty for Drug Trafficking in the United States, Singapore, Malaysia, Indonesia and Thailand: A Comparative Legal Analysis," Laws, MDPI, vol. 2(2), pages 1-35, June.
    32. Byeolbee Um, 2018. "Ego-Resiliency, Parenting Style, and Peer Attachment as Predictors of South Korean Middle School Students’ School Adjustment," International Journal of Social Science Studies, Redfame publishing, vol. 6(7), pages 14-30, July.
    33. Takayuki Hayashi & Yuko Fujigaki, 1999. "Differences in knowledge production between disciplines based on analysis of paper styles and citation patterns," Scientometrics, Springer;Akadémiai Kiadó, vol. 46(1), pages 73-86, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03764-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.