IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/3001107.html

Assessment of transparency indicators across the biomedical literature: How open is open?

Author

Listed:
  • Stylianos Serghiou
  • Despina G Contopoulos-Ioannidis
  • Kevin W Boyack
  • Nico Riedel
  • Joshua D Wallach
  • John P A Ioannidis

Abstract

Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.This study uses novel open source automated tools to monitor transparency across all 2.75 million open access articles on PubMed Central, discovering that different disciplines, journals and publishers abide by principles of transparency to varying degrees over time.

Suggested Citation

  • Stylianos Serghiou & Despina G Contopoulos-Ioannidis & Kevin W Boyack & Nico Riedel & Joshua D Wallach & John P A Ioannidis, 2021. "Assessment of transparency indicators across the biomedical literature: How open is open?," PLOS Biology, Public Library of Science, vol. 19(3), pages 1-26, March.
    • Handle: RePEc:plo:pbio00:3001107
    DOI: 10.1371/journal.pbio.3001107
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001107
    Download Restriction: no
    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.3001107&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pbio.3001107?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. C. Glenn Begley & Lee M. Ellis, 2012. "Raise standards for preclinical cancer research," Nature, Nature, vol. 483(7391), pages 531-533, March.
    2. John P A Ioannidis, 2014. "How to Make More Published Research True," PLOS Medicine, Public Library of Science, vol. 11(10), pages 1-6, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eero Raittio & Ahmad Sofi-Mahmudi & Erfan Shamsoddin, 2022. "The use of the phrase “data not shown” in dental research," PLOS ONE, Public Library of Science, vol. 17(8), pages 1-11, August.
    2. Klebel, Thomas & Bianchi, Federico & Ross-Hellauer, Tony & Squazzoni, Flaminio, 2025. "The paradox of competition: How funding models could undermine the uptake of data sharing practices," Research Policy, Elsevier, vol. 54(10).
    3. Emmanuel A Zavalis & John P A Ioannidis, 2022. "A meta-epidemiological assessment of transparency indicators of infectious disease models," PLOS ONE, Public Library of Science, vol. 17(10), pages 1-13, October.
    4. Gabriel Gonçalves da Costa & Kleber Neves & Olavo Amaral, 2024. "Estimating the replicability of highly cited clinical research (2004–2018)," PLOS ONE, Public Library of Science, vol. 19(8), pages 1-24, August.
    5. Giovanni Colavizza & Lauren Cadwallader & Marcel LaFlamme & Grégory Dozot & Stéphane Lecorney & Daniel Rappo & Iain Hrynaszkiewicz, 2024. "An analysis of the effects of sharing research data, code, and preprints on citations," PLOS ONE, Public Library of Science, vol. 19(10), pages 1-19, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kiri, Bralind & Lacetera, Nicola & Zirulia, Lorenzo, 2018. "Above a swamp: A theory of high-quality scientific production," Research Policy, Elsevier, vol. 47(5), pages 827-839.
    2. Hussinger, Katrin & Pellens, Maikel, 2019. "Guilt by association: How scientific misconduct harms prior collaborators," Research Policy, Elsevier, vol. 48(2), pages 516-530.
    3. Yan Li & Xiang Zhou & Rui Chen & Xianyang Zhang & Hongyuan Cao, 2024. "STAREG: Statistical replicability analysis of high throughput experiments with applications to spatial transcriptomic studies," PLOS Genetics, Public Library of Science, vol. 20(10), pages 1-19, October.
    4. Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
    5. Felix Holzmeister & Magnus Johannesson & Colin F. Camerer & Yiling Chen & Teck-Hua Ho & Suzanne Hoogeveen & Juergen Huber & Noriko Imai & Taisuke Imai & Lawrence Jin & Michael Kirchler & Alexander Ly , 2025. "Examining the replicability of online experiments selected by a decision market," Nature Human Behaviour, Nature, vol. 9(2), pages 316-330, February.
    6. Marlo M Vernon & E Andrew Balas & Shaher Momani, 2018. "Are university rankings useful to improve research? A systematic review," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    7. Malika Ihle & Isabel S. Winney & Anna Krystalli & Michael Croucher, 2017. "Striving for transparent and credible research: practical guidelines for behavioral ecologists," Behavioral Ecology, International Society for Behavioral Ecology, vol. 28(2), pages 348-354.
    8. Baltussen, Guido & Swinkels, Laurens & Van Vliet, Pim, 2021. "Global factor premiums," Journal of Financial Economics, Elsevier, vol. 142(3), pages 1128-1154.
    9. Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
    10. Fecher, Benedikt & Fräßdorf, Mathis & Hebing, Marcel & Wagner, Gert G., 2017. "Replikationen, Reputation und gute wissenschaftliche Praxis," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 68(2-3), pages 154-158.
    11. Ádám Kun, 2018. "Publish and Who Should Perish: You or Science?," Publications, MDPI, vol. 6(2), pages 1-16, April.
    12. Andrea Saltelli & Monica Fiore, 2020. "From sociology of quantification to ethics of quantification," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 7(1), pages 1-8, December.
    13. Schweinsberg, Martin & Feldman, Michael & Staub, Nicola & van den Akker, Olmo R. & van Aert, Robbie C.M. & van Assen, Marcel A.L.M. & Liu, Yang & Althoff, Tim & Heer, Jeffrey & Kale, Alex & Mohamed, Z, 2021. "Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 228-249.
    14. Susanne Wieschowski & Diego S Silva & Daniel Strech, 2016. "Animal Study Registries: Results from a Stakeholder Analysis on Potential Strengths, Weaknesses, Facilitators, and Barriers," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-12, November.
    15. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    16. Francisco Rowe & Gunther Maier & Daniel Arribas-Bel & Sergio Rey, 2020. "The Potential of Notebooks for Scientific Publication, Reproducibility and Dissemination," REGION, European Regional Science Association, vol. 7, pages 1-5.
    17. Christoph Semken & David Rossell, 2022. "Specification analysis for technology use and teenager well‐being: Statistical validity and a Bayesian proposal," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1330-1355, November.
    18. Salandra, Rossella & Criscuolo, Paola & Salter, Ammon, 2021. "Directing scientists away from potentially biased publications: the role of systematic reviews in health care," Research Policy, Elsevier, vol. 50(1).
    19. Yu Zhang & Bingwen Yang & Ying Yu & Xia Wang & Chunyan Niu & Yongzhuo Zhang & Yang Liu & Jingshu Li & Caihang Zhang & Jiayi Yang & Jiayu Tian & Zheng Liu & Zhiyu Tang & Yunhua Gao & Yuanting Zheng & Y, 2026. "A metrological foundation for absolute transcriptomics using International System of Units-anchored calibrators," Nature Communications, Nature, vol. 17(1), pages 1-19, December.
    20. Vuong, Quan-Hoang, 2017. "Open data, open review and open dialogue in making social sciences plausible," OSF Preprints du8tj, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:3001107. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.