IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0066844.html
   My bibliography  Save this article

Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review

Author

Listed:
  • Kerry Dwan
  • Carrol Gamble
  • Paula R Williamson
  • Jamie J Kirkham
  • the Reporting Bias Group

Abstract

Background: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Methodology/Principal Findings: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40–62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. Conclusions: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Suggested Citation

  • Kerry Dwan & Carrol Gamble & Paula R Williamson & Jamie J Kirkham & the Reporting Bias Group, 2013. "Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review," PLOS ONE, Public Library of Science, vol. 8(7), pages 1-37, July.
    • Handle: RePEc:plo:pone00:0066844
    DOI: 10.1371/journal.pone.0066844
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0066844
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0066844&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0066844?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Je-Young Lee & Minkyung Baek, 2023. "Effects of Gamification on Students’ English Language Proficiency: A Meta-Analysis on Research in South Korea," Sustainability, MDPI, vol. 15(14), pages 1-19, July.
    2. Claire Godard-Sebillotte & Mélanie Le Berre & Tibor Schuster & Miguel Trottier & Isabelle Vedel, 2019. "Impact of health service interventions on acute hospital use in community-dwelling persons with dementia: A systematic literature review and meta-analysis," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    3. Eijgermans, D.G.M. & Fang, Y. & Jansen, D.E.M.C. & Bramer, W.M. & Raat, H. & Jansen, W., 2021. "Individual and contextual determinants of children’s and adolescents’ mental health care use: A systematic review," Children and Youth Services Review, Elsevier, vol. 131(C).
    4. Chuan Hong & Georgia Salanti & Sally C. Morton & Richard D. Riley & Haitao Chu & Stephen E. Kimmel & Yong Chen, 2020. "Testing small study effects in multivariate meta‐analysis," Biometrics, The International Biometric Society, vol. 76(4), pages 1240-1250, December.
    5. Kellia Chiu & Quinn Grundy & Lisa Bero, 2017. "‘Spin’ in published biomedical literature: A methodological systematic review," PLOS Biology, Public Library of Science, vol. 15(9), pages 1-16, September.
    6. Dengsheng Wu & Huidong Wu & Jianping Li, 2024. "Citation advantage of positive words: predictability, temporal evolution, and universality in varied quality journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4275-4293, July.
    7. Arnaud Vaganay, 2016. "Outcome Reporting Bias in Government-Sponsored Policy Evaluations: A Qualitative Content Analysis of 13 Studies," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-21, September.
    8. Anette Blümle & Tobias Haag & James Balmford & Gerta Rücker & Martin Schumacher & Nadine Binder, 2020. "A multi-state model analysis of the time from ethical approval to publication of clinical research studies," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-15, March.
    9. E. Decullier & P. V. Tang & L. Huot & H. Maisonneuve, 2021. "Why an automated tracker finds poor sharing of clinical trial results for an academic sponsor: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1239-1248, February.
    10. Jamie J Kirkham & Kerry M Dwan & Anette Blümle & Erik von Elm & Paula R Williamson, 2016. "How Much Participant Outcome Data Is Missing from Sight: Findings from a Cohort of Trials Submitted to a German Research Ethics Committee," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-8, June.
    11. John Copas, 2022. "Akaike Memorial Lecture 2020: Some of the challenges of statistical applications," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(4), pages 615-637, August.
    12. Salandra, Rossella & Criscuolo, Paola & Salter, Ammon, 2021. "Directing scientists away from potentially biased publications: the role of systematic reviews in health care," Research Policy, Elsevier, vol. 50(1).
    13. Jaithri Ananthapavan & Gary Sacks & Marj Moodie & Rob Carter, 2014. "Economics of Obesity — Learning from the Past to Contribute to a Better Future," IJERPH, MDPI, vol. 11(4), pages 1-19, April.
    14. Zhou-min Yuan & Mingxin Yao, 2022. "Is academic writing becoming more positive? A large-scale diachronic case study of Science research articles across 25 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6191-6207, November.
    15. Tinashe Dune & Jacqueline Ullman & Tania Ferfolja & Jack Thepsourinthone & Shirali Garga & Zelalem Mengesha, 2020. "Are Services Inclusive? A Review of the Experiences of Older GSD Women in Accessing Health, Social and Aged Care Services," IJERPH, MDPI, vol. 17(11), pages 1-17, May.
    16. Martin E Héroux & Janet L Taylor & Simon C Gandevia, 2015. "The Use and Abuse of Transcranial Magnetic Stimulation to Modulate Corticospinal Excitability in Humans," PLOS ONE, Public Library of Science, vol. 10(12), pages 1-10, December.
    17. Xueying Liu & Haoran Zhu, 2023. "Linguistic positivity in soft and hard disciplines: temporal dynamics, disciplinary variation, and the relationship with research impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 3107-3127, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0066844. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.