IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i1d10.1007_s11192-020-03764-3.html
   My bibliography  Save this article

Number of references: a large-scale study of interval ratios

Author

Listed:
  • Jeppe Nicolaisen

    (University of Copenhagen)

  • Tove Faber Frandsen

    (University of Southern Denmark)

Abstract

The paper presents a large-scale study (covering 26,998,764 items) of the development in the number of references over time (1996–2019) in three document types (articles; reviews; notes) from seven fields (Arts and Humanities; Social Sciences; Computer Science; Mathematics; Engineering; Medicine; Physics and Astronomy). Using interval ratios instead of average numbers, the paper makes it possible to follow the development, and to locate the main causes of growth in the number of references over time. The results show significant differences between fields and document types. The number of references in journal articles and reviews are growing in all fields (except for the reviews in Arts and Humanities that remain stable over time), but at different pace; The number of references in notes are growing in some fields (again at different pace) and are stable in others. The observed growth is primarily caused by a drop in short reference lists and a corresponding increase in a bit longer and medium size reference lists. Long and very long reference lists remain much more stable in shares over time, and does therefore not contribute much to the observed growth. The results underline the importance of normalizing citation data, and for taking citation inflation into account when conducting citation analyses expanding different fields, document types, and longer time-periods.

Suggested Citation

  • Jeppe Nicolaisen & Tove Faber Frandsen, 2021. "Number of references: a large-scale study of interval ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 259-285, January.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03764-3
    DOI: 10.1007/s11192-020-03764-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03764-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03764-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John Mingers & Martin Meyer, 2017. "Normalizing Google Scholar data for use in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 1111-1121, August.
    2. Iñaki Ucar & Felipe López-Fernandino & Pablo Rodriguez-Ulibarri & Laura Sesma-Sanchez & Veronica Urrea-Micó & Joaquín Sevilla, 2014. "Growth in the number of references in engineering journal papers during the 1972–2013 period," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1855-1864, March.
    3. Pan, Raj K. & Petersen, Alexander M. & Pammolli, Fabio & Fortunato, Santo, 2018. "The memory of science: Inflation, myopia, and the knowledge network," Journal of Informetrics, Elsevier, vol. 12(3), pages 656-678.
    4. John Mingers & Martin Meyer, 2017. "Erratum to: Normalizing Google Scholar data for use in research evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 1123-1124, August.
    5. Zitt, Michel, 2010. "Citing-side normalization of journal impact: A robust variant of the Audience Factor," Journal of Informetrics, Elsevier, vol. 4(3), pages 392-406.
    6. Helmut A. Abt & Eugene Garfield, 2002. "Is the relationship between numbers of references and paper lengths the same for all sciences?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 53(13), pages 1106-1112, November.
    7. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    8. Petersen, Alexander M. & Pan, Raj K. & Pammolli, Fabio & Fortunato, Santo, 2019. "Methods to account for citation inflation in research evaluation," Research Policy, Elsevier, vol. 48(7), pages 1855-1865.
    9. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 2.
    10. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
    11. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    12. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    13. Günter Krampen, 2010. "Acceleration of citing behavior after the millennium? Exemplary bibliometric reference analyses for psychology journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 507-513, May.
    14. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    15. Erwin Krauskopf, 2019. "Missing documents in Scopus: the case of the journal Enfermeria Nefrologica," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 543-547, April.
    16. Bornmann, Lutz & Marx, Werner, 2018. "Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics," Journal of Informetrics, Elsevier, vol. 12(3), pages 598-604.
    17. Sánchez-Gil, Susana & Gorraiz, Juan & Melero-Fuentes, David, 2018. "Reference density trends in the major disciplines," Journal of Informetrics, Elsevier, vol. 12(1), pages 42-58.
    18. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    19. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    20. Ben‐Ami Lipetz, 1999. "Aspects of JASIS authorship through five decades," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(11), pages 994-1003.
    21. Editorial Article, 0. "Contents," Economics of Contemporary Russia, Regional Public Organization for Assistance to the Development of Institutions of the Department of Economics of the Russian Academy of Sciences, issue 3.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shing-Yun Jung & Ting-Han Lin & Chia-Hung Liao & Shyan-Ming Yuan & Chuen-Tsai Sun, 2022. "Intent-Controllable Citation Text Generation," Mathematics, MDPI, vol. 10(10), pages 1-17, May.
    2. Katchanov, Yurij L. & Markova, Yulia V. & Shmatko, Natalia A., 2023. "Uncited papers in the structure of scientific communication," Journal of Informetrics, Elsevier, vol. 17(2).
    3. Tove Faber Frandsen & Jeppe Nicolaisen, 2023. "Defining the unscholarly publication: a bibliometric study of uncited and barely cited publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1337-1350, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    2. Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
    3. Dennis Coates & Brad R. Humphreys, 2008. "Do Economists Reach a Conclusion on Subsidies for Sports Franchises, Stadiums, and Mega-Events?," Econ Journal Watch, Econ Journal Watch, vol. 5(3), pages 294-315, September.
    4. Blair Jenkins, 2009. "Rent Control: Do Economists Agree?," Econ Journal Watch, Econ Journal Watch, vol. 6(1), pages 73-112, January.
    5. Ann Mari May, 2008. "On Gender Balance in the Economics Profession," Econ Journal Watch, Econ Journal Watch, vol. 5(2), pages 193-198, May.
    6. Peter Hennecke, 2021. "The ECB’s New Monetary Policy Strategy," Intereconomics: Review of European Economic Policy, Springer;ZBW - Leibniz Information Centre for Economics;Centre for European Policy Studies (CEPS), vol. 56(5), pages 295-298, September.
    7. Toma Lankauskiene, 2021. "Labour Productivity Growth Determinants in the Manufacturing Sector in the Baltic States," ConScienS Conference Proceedings 025tl, Research Association for Interdisciplinary Studies.
    8. Lars P Feld & Volker Wieland, 2021. "The German Federal Constitutional Court Ruling and the European Central Bank’s Strategy," Journal of Financial Regulation, Oxford University Press, vol. 7(2), pages 217-253.
    9. N��ez Ferrer, Jorge, 2021. "Avoiding the Main Risks in the Recovery Plans of Member States," CEPS Papers 32463, Centre for European Policy Studies.
    10. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    11. Garett Jones, 2008. "What Is the Right Number of Women? Hints and Puzzles from Cognitive Ability Research," Econ Journal Watch, Econ Journal Watch, vol. 5(2), pages 227-239, May.
    12. Lars Jonung & Eoin Drea, 2010. "It Can't Happen, It's a Bad Idea, It Won't Last: U.S. Economists on the EMU and the Euro, 1989–2002," Econ Journal Watch, Econ Journal Watch, vol. 7(1), pages 1-4–52, January.
    13. Ian Ayres & John J. Donohue, 2009. "Yet Another Refutation of the More Guns, Less Crime Hypothesis—With Some Help From Moody and Marvell," Econ Journal Watch, Econ Journal Watch, vol. 6(1), pages 35-59, January.
    14. Richard Jong-A-Pin & Jakob De Haan, 2008. "Growth Accelerations and Regime Changes: A Correction," Econ Journal Watch, Econ Journal Watch, vol. 5(1), pages 51-58, January.
    15. E. Frank Stephenson & Erin E. Wendt, 2009. "Occupational Licensing: Scant Treatment in Labor Texts," Econ Journal Watch, Econ Journal Watch, vol. 6(2), pages 181-194, May.
    16. Kjerstin Tevik & Geir Selbæk & Knut Engedal & Arnfinn Seim & Steinar Krokstad & Anne-S Helvik, 2019. "Mortality in older adults with frequent alcohol consumption and use of drugs with addiction potential – The Nord Trøndelag Health Study 2006-2008 (HUNT3), Norway, a population-based study," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-24, April.
    17. Yoshihisa Amae & Jens Damm, 2011. "“Whither Taiwanization?†State, Society and Cultural Production in the New Era," Journal of Current Chinese Affairs - China aktuell, Institute of Asian Studies, GIGA German Institute of Global and Area Studies, Hamburg, vol. 40(1), pages 3-17.
    18. Yunhee Park & Hyun-Jung Yun, 2020. "A Multilevel Investigation of Fall Prevention Behavior Among Nursing Staff of South Korean Geriatric Hospitals," Global Journal of Health Science, Canadian Center of Science and Education, vol. 12(10), pages 1-97, September.
    19. Arthur M. Diamond, Jr., 2007. "Thriving at Amazon: How Schumpeter Lives in Books Today," Econ Journal Watch, Econ Journal Watch, vol. 4(3), pages 338-344, September.
    20. Paul Trampe, 2008. "The EITC Disincentive: A Reply to Dr. Hilary Hoynes," Econ Journal Watch, Econ Journal Watch, vol. 5(1), pages 59-65, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03764-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.