IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0180986.html
   My bibliography  Save this article

Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

Author

Listed:
  • Benjamin Howard
  • Jared T Scott
  • Mark Blubaugh
  • Brie Roepke
  • Caleb Scheckel
  • Matt Vassar

Abstract

Background: Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals. Methods: We searched PubMed for randomized controlled trials from Jan 1, 2010 –Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined. Results: 180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results. Conclusion: Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

Suggested Citation

  • Benjamin Howard & Jared T Scott & Mark Blubaugh & Brie Roepke & Caleb Scheckel & Matt Vassar, 2017. "Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-14, July.
    • Handle: RePEc:plo:pone00:0180986
    DOI: 10.1371/journal.pone.0180986
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0180986
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0180986&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0180986?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kerry Dwan & Douglas G Altman & Juan A Arnaiz & Jill Bloom & An-Wen Chan & Eugenia Cronin & Evelyne Decullier & Philippa J Easterbrook & Erik Von Elm & Carrol Gamble & Davina Ghersi & John P A Ioannid, 2008. "Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias," PLOS ONE, Public Library of Science, vol. 3(8), pages 1-31, August.
    2. Christine Schmucker & Lisa K Schell & Susan Portalupi & Patrick Oeller & Laura Cabrera & Dirk Bassler & Guido Schwarzer & Roberta W Scherer & Gerd Antes & Erik von Elm & Joerg J Meerpohl & on behalf o, 2014. "Extent of Non-Publication in Cohorts of Studies Approved by Research Ethics Committees or Included in Trial Registries," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-25, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Matthew Thomas Sims & Byron Nice Detweiler & Jared Thomas Scott & Benjamin McKinnley Howard & Grant Richard Detten & Matt Vassar, 2017. "Inconsistent selection of outcomes and measurement devices found in shoulder arthroplasty research: An analysis of studies on ClinicalTrials.gov," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-16, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schaub, Jason & Stander, Willem J. & Montgomery, Paul, 2022. "LGBTQ+ Young People’s Health and Well-being Experiences in Out-of-home Social Care: A scoping review," Children and Youth Services Review, Elsevier, vol. 143(C).
    2. Paolo Crosetto & Antonio Filippin & Janna Heider, 2015. "A Study of Outcome Reporting Bias Using Gender Differences in Risk Attitudes," CESifo Economic Studies, CESifo Group, vol. 61(1), pages 239-262.
    3. Nikolaos Pandis & Padhraig S Fleming & Helen Worthington & Kerry Dwan & Georgia Salanti, 2015. "Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-10, September.
    4. Joseph S Ross & Gregory K Mulvey & Elizabeth M Hines & Steven E Nissen & Harlan M Krumholz, 2009. "Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis," PLOS Medicine, Public Library of Science, vol. 6(9), pages 1-9, September.
    5. Wynanda A van Enst & Rob J P M Scholten & Lotty Hooft, 2012. "Identification of Additional Trials in Prospective Trial Registers for Cochrane Systematic Reviews," PLOS ONE, Public Library of Science, vol. 7(8), pages 1-5, August.
    6. Ewelina Rogozińska & Elizabeth Gargon & Rocío Olmedo-Requena & Amani Asour & Natalie A M Cooper & Claire L Vale & Janneke van’t Hooft, 2020. "Methods used to assess outcome consistency in clinical studies: A literature-based evaluation," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-12, July.
    7. Elizabeth Gargon & Binu Gurung & Nancy Medley & Doug G Altman & Jane M Blazeby & Mike Clarke & Paula R Williamson, 2014. "Choosing Important Health Outcomes for Comparative Effectiveness Research: A Systematic Review," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-12, June.
    8. Jamie J Kirkham & Doug G Altman & Paula R Williamson, 2010. "Bias Due to Changes in Specified Outcomes during the Systematic Review Process," PLOS ONE, Public Library of Science, vol. 5(3), pages 1-5, March.
    9. Daniele Fanelli, 2010. "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data," PLOS ONE, Public Library of Science, vol. 5(4), pages 1-7, April.
    10. Dongying Li & Tess Menotti & Yizhen Ding & Nancy M. Wells, 2021. "Life Course Nature Exposure and Mental Health Outcomes: A Systematic Review and Future Directions," IJERPH, MDPI, vol. 18(10), pages 1-28, May.
    11. Amin Haghnejad & Mahboobeh Farahati, 2024. "An overview of meta-analytic methods for economic research," Papers 2412.10608, arXiv.org, revised Mar 2025.
    12. Thomas Pfeiffer & Lars Bertram & John P A Ioannidis, 2011. "Quantifying Selective Reporting and the Proteus Phenomenon for Multiple Datasets with Similar Bias," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-7, March.
    13. Ana Virgolino & Osvaldo Santos & Joana Costa & Mónica Fialho & Ivo Iavicoli & Tiina Santonen & Hanna Tolonen & Evangelia Samoli & Klea Katsouyanni & Georgios Baltatzis & Flavia Ruggieri & Annalisa Abb, 2021. "Challenges to Evidence Synthesis and Identification of Data Gaps in Human Biomonitoring," IJERPH, MDPI, vol. 18(6), pages 1-11, March.
    14. Michal Kicinski, 2013. "Publication Bias in Recent Meta-Analyses," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    15. Carl Berning & Bernd Weiß, 2016. "Publication bias in the German social sciences: an application of the caliper test to three top-tier German social science journals," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(2), pages 901-917, March.
    16. Erick H Turner & Andrea Cipriani & Toshi A Furukawa & Georgia Salanti & Ymkje Anna de Vries, 2022. "Selective publication of antidepressant trials and its influence on apparent efficacy: Updated comparisons and meta-analyses of newer versus older trials," PLOS Medicine, Public Library of Science, vol. 19(1), pages 1-21, January.
    17. Thomas J. Kniesner & W. Kip Viscusi, 2023. "Compensating Differentials for Occupational Health and Safety Risks: Implications of Recent Evidence," Research in Labor Economics, in: 50th Celebratory Volume, volume 50, pages 83-116, Emerald Group Publishing Limited.
    18. Carina Benstoem & Ajay Moza & Rüdiger Autschbach & Christian Stoppe & Andreas Goetzenich, 2015. "Evaluating Outcomes Used in Cardiothoracic Surgery Interventional Research: A Systematic Review of Reviews to Develop a Core Outcome Set," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    19. Mangirdas Morkunas & Elzė Rudienė & Lukas Giriūnas & Laura Daučiūnienė, 2020. "Assessment of Factors Causing Bias in Marketing- Related Publications," Publications, MDPI, vol. 8(4), pages 1-16, October.
    20. Michał Krawczyk, 2015. "The Search for Significance: A Few Peculiarities in the Distribution of P Values in Experimental Psychology Literature," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-19, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0180986. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.