IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/aqyhg.html
   My bibliography  Save this paper

Meta-Research: How problematic citing practices distort science

Author

Listed:
  • Horbach, Serge
  • Aagaard, Kaare
  • Schneider, Jesper W.

    (Aarhus University)

Abstract

Citing practices constitute a core element in scientific research and communication in which they serve several important functions. They both comprise the principal unit of science’s social reward system and they establish epistemic genealogy, showing the foundations on which claims are built. Distorted or problematic citing behaviour can hence have major harmful consequences for the well-functioning of the scientific enterprise. In this article we employ a case study approach to show how, even in the field of research integrity, or more broadly, meta-research, researchers commonly adhere to suboptimal citing practices. Our findings highlight, among other shortcomings, an apparent lack of critical engagement with the cited literature, leading to incorrect reproductions of claims and overgeneralisation of supposed research findings. We link such problematic citing practices to the state of crisis that science is currently claimed to face, discussing how these practices might be both a symptom and a cause of these crises.

Suggested Citation

  • Horbach, Serge & Aagaard, Kaare & Schneider, Jesper W., 2021. "Meta-Research: How problematic citing practices distort science," MetaArXiv aqyhg, Center for Open Science.
  • Handle: RePEc:osf:metaar:aqyhg
    DOI: 10.31219/osf.io/aqyhg
    as

    Download full text from publisher

    File URL: https://osf.io/download/6033c54f34d304043fe3c4d3/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/aqyhg?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Melissa S. Anderson & Emily A. Ronning & Raymond De Vries & Brian C. Martinson, 2010. "Extending the Mertonian Norms: Scientists' Subscription to Norms of Research," The Journal of Higher Education, Taylor & Francis Journals, vol. 81(3), pages 366-393, May.
    2. Jesper W. Schneider, 2006. "Concept symbols revisited: Naming clusters by parsing and filtering of noun phrases from citation contexts of concept symbols," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 573-593, September.
    3. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom-cited influences," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    4. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    5. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    6. María‐del‐Mar Camacho‐Miñano & Manuel Núñez‐Nickel, 2009. "The multilayered nature of reference selection," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(4), pages 754-777, April.
    7. Ewen Callaway, 2011. "Report finds massive fraud at Dutch universities," Nature, Nature, vol. 479(7371), pages 15-15, November.
    8. Jenny T van der Steen & Cornelis A van den Bogert & Mirjam C van Soest-Poortvliet & Soulmaz Fazeli Farsani & René H J Otten & Gerben ter Riet & Lex M Bouter, 2018. "Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-15, February.
    9. M.H. MacRoberts & B.R. MacRoberts, 2010. "Problems of citation analysis: A study of uncited and seldom‐cited influences," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 1-12, January.
    10. Norman Kaplan, 1965. "The norms of citation behavior: Prolegomena to the footnote," American Documentation, Wiley Blackwell, vol. 16(3), pages 179-184, July.
    11. Marcus R. Munafò & Brian A. Nosek & Dorothy V. M. Bishop & Katherine S. Button & Christopher D. Chambers & Nathalie Percie du Sert & Uri Simonsohn & Eric-Jan Wagenmakers & Jennifer J. Ware & John P. A, 2017. "A manifesto for reproducible science," Nature Human Behaviour, Nature, vol. 1(1), pages 1-9, January.
    12. Ziming Liu, 1997. "Citation theories in the framework of international flow of information: New evidence with translation analysis," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(1), pages 80-87, January.
    13. Henry Small, 2004. "On the shoulders of Robert Merton: Towards a normative theory of citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(1), pages 71-79, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Teplitskiy, Misha & Duede, Eamon & Menietti, Michael & Lakhani, Karim R., 2022. "How status of research papers affects the way they are read and cited," Research Policy, Elsevier, vol. 51(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ehsan Mohammadi & Mike Thelwall, 2013. "Assessing non-standard article impact using F1000 labels," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 383-395, November.
    2. Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    3. Marc Bertin & Iana Atanassova & Cassidy R. Sugimoto & Vincent Lariviere, 2016. "The linguistic patterns and rhetorical structure of citation context: an approach using n-grams," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1417-1434, December.
    4. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    5. Sten F Odenwald, 2020. "A citation study of earth science projects in citizen science," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    6. Hu, Zewen & Wu, Yishan, 2014. "Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals," Journal of Informetrics, Elsevier, vol. 8(1), pages 136-146.
    7. Lawrence Smolinsky & Aaron Lercher, 2012. "Citation rates in mathematics: a study of variation by subdiscipline," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 911-924, June.
    8. Tobias Opthof & Loet Leydesdorff, 2011. "A comment to the paper by Waltman et al., Scientometrics, 87, 467–481, 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 1011-1016, September.
    9. Lutz Bornmann & Robin Haunschild, 2017. "Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 937-943, February.
    10. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    11. Jianhua Hou & Jiantao Ye, 2020. "Are uncited papers necessarily all nonimpact papers? A quantitative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1631-1662, August.
    12. Bornmann, Lutz & Haunschild, Robin & Mutz, Rüdiger, 2020. "Should citations be field-normalized in evaluative bibliometrics? An empirical analysis based on propensity score matching," Journal of Informetrics, Elsevier, vol. 14(4).
    13. Mike Thelwall, 2019. "The influence of highly cited papers on field normalised indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 519-537, February.
    14. Michael Calver & Kate Bryant & Grant Wardell-Johnson, 2018. "Quantifying the internationality and multidisciplinarity of authors and journals using ecological statistics," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 731-748, May.
    15. Isabel Bernal, 2013. "Open Access and the Changing Landscape of Research Impact Indicators: New Roles for Repositories," Publications, MDPI, vol. 1(2), pages 1-22, July.
    16. Katherine W. McCain, 2012. "Assessing Obliteration by Incorporation: Issues and Caveats," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(11), pages 2129-2139, November.
    17. Bornmann, Lutz & Schier, Hermann & Marx, Werner & Daniel, Hans-Dieter, 2012. "What factors determine citation counts of publications in chemistry besides their quality?," Journal of Informetrics, Elsevier, vol. 6(1), pages 11-18.
    18. Zhi Li & Qinke Peng & Che Liu, 2016. "Two citation-based indicators to measure latent referential value of papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1299-1313, September.
    19. Christin Katharina Kreutz & Premtim Sahitaj & Ralf Schenkel, 2020. "Evaluating semantometrics from computer science publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2915-2954, December.
    20. Jiang, Zhuoren & Lin, Tianqianjin & Huang, Cui, 2023. "Deep representation learning of scientific paper reveals its potential scholarly impact," Journal of Informetrics, Elsevier, vol. 17(1).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:aqyhg. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.