IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0000332.html
   My bibliography  Save this article

Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial

Author

Listed:
  • Erik Cobo
  • Albert Selva-O'Callagham
  • Josep-Maria Ribera
  • Francesc Cardellach
  • Ruth Dominguez
  • Miquel Vilardell

Abstract

Background: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. Methodology and Principal Findings: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with “no statistical expert” and “no checklist” as controls. The two interventions were crossed in a 2×2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6–24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: −0.3–+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3–6.7), showing a significant improvement in quality. Conclusions and Significance: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.

Suggested Citation

  • Erik Cobo & Albert Selva-O'Callagham & Josep-Maria Ribera & Francesc Cardellach & Ruth Dominguez & Miquel Vilardell, 2007. "Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial," PLOS ONE, Public Library of Science, vol. 2(3), pages 1-8, March.
  • Handle: RePEc:plo:pone00:0000332
    DOI: 10.1371/journal.pone.0000332
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000332
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0000332&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0000332?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Des Jarlais, D.C. & Lyles, C. & Crepaz, N., 2004. "Improving the Reporting Quality of Nonrandomized Evaluations of Behavioral and Public Health Interventions: The TREND Statement," American Journal of Public Health, American Public Health Association, vol. 94(3), pages 361-366.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ulrike Held & Klaus Steigmiller & Michael Hediger & Martina Gosteli & Kelly A Reeve & Stefanie von Felten & Eva Furrer, 2020. "Is reporting quality in medical publications associated with biostatisticians as co-authors? A registered report protocol," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-9, November.
    2. Mina Moradzadeh & Shahram Sedghi & Sirous Panahi, 2023. "Towards a new paradigm for ‘journal quality’ criteria: a scoping review," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 279-321, January.
    3. Gastón Sanglier & Roberto A. Gónzález & Sonia Cesteros & Eduardo J. López, 2020. "Validation of a Mathematical Model Applied to Four Autonomous Communities in Spain to Determine the Number of People Infected by Covid-19," Modern Applied Science, Canadian Center of Science and Education, vol. 14(6), pages 110-110, June.
    4. Alfonso Ibáñez & Pedro Larrañaga & Concha Bielza, 2013. "Cluster methods for assessing research performance: exploring Spanish computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 571-600, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giuseppe La Torre & Remigio Bova & Rosario Andrea Cocchiara & Cristina Sestili & Anna Tagliaferri & Simona Maggiacomo & Camilla Foschi & William Zomparelli & Maria Vittoria Manai & David Shaholli & Va, 2023. "What Are the Determinants of the Quality of Systematic Reviews in the International Journals of Occupational Medicine? A Methodological Study Review of Published Literature," IJERPH, MDPI, vol. 20(2), pages 1-12, January.
    2. Penelope Love & Jillian Whelan & Colin Bell & Jane McCracken, 2019. "Measuring Rural Food Environments for Local Action in Australia: A Systematic Critical Synthesis Review," IJERPH, MDPI, vol. 16(13), pages 1-21, July.
    3. Paul Montgomery & Caitlin R Ryus & Catherine S Dolan & Sue Dopson & Linda M Scott, 2012. "Sanitary Pad Interventions for Girls' Education in Ghana: A Pilot Study," PLOS ONE, Public Library of Science, vol. 7(10), pages 1-7, October.
    4. Wai Quin Ng & Jane Neill, 2006. "Evidence for early oral feeding of patients after elective open colorectal surgery: a literature review," Journal of Clinical Nursing, John Wiley & Sons, vol. 15(6), pages 696-709, June.
    5. Natalia Stanulewicz & Emily Knox & Melanie Narayanasamy & Noureen Shivji & Kamlesh Khunti & Holly Blake, 2019. "Effectiveness of Lifestyle Health Promotion Interventions for Nurses: A Systematic Review," IJERPH, MDPI, vol. 17(1), pages 1-36, December.
    6. Zhichao Jin & Danghui Yu & Luoman Zhang & Hong Meng & Jian Lu & Qingbin Gao & Yang Cao & Xiuqiang Ma & Cheng Wu & Qian He & Rui Wang & Jia He, 2010. "A Retrospective Survey of Research Design and Statistical Analyses in Selected Chinese Medical Journals in 1998 and 2008," PLOS ONE, Public Library of Science, vol. 5(5), pages 1-4, May.
    7. Bastiaanssen, Inge L.W. & Delsing, Marc J.M.H. & Kroes, Gert & Engels, Rutger C.M.E. & Veerman, Jan W., 2014. "Group care worker interventions and child problem behavior in residential youth care: Course and bidirectional associations," Children and Youth Services Review, Elsevier, vol. 39(C), pages 48-56.
    8. Winter, James C. & Darmstadt, Gary L. & Davis, Jennifer, 2021. "The role of piped water supplies in advancing health, economic development, and gender equality in rural communities," Social Science & Medicine, Elsevier, vol. 270(C).
    9. Shu‐Fang Chang & Shu‐Ching Chiu, 2020. "Effect of resistance training on quality of life in older people with sarcopenic obesity living in long‐term care institutions: A quasi‐experimental study," Journal of Clinical Nursing, John Wiley & Sons, vol. 29(13-14), pages 2544-2556, July.
    10. Eui Geum Oh & Jeong Hyun Kim & Hyun Joo Lee, 2019. "Effects of a safe transition programme for discharged patients with high unmet needs," Journal of Clinical Nursing, John Wiley & Sons, vol. 28(11-12), pages 2319-2328, June.
    11. repec:zbw:rwirep:0290 is not listed on IDEAS
    12. Shu Fen Chen & Peng-Hui Wang & Shu-Chen Kuo & Yin-Chen Chen & Huei-Jhen Sia & Pei-Hsuan Lee & Jia-Hwa Yang & Senyeong Kao, 2022. "Early and Standard Urinary Catheter Removal After Gynecological Surgery for Benign Lesions: A Quasi-Experimental Study," Clinical Nursing Research, , vol. 31(3), pages 489-496, March.
    13. Lilian Salm & Dimitri Chapalley & Stéphanie Fabienne Perrodin & Franziska Tschan & Daniel Candinas & Guido Beldi, 2020. "Impact of changing the surgical team for wound closure on surgical site infection: A matched case-control study," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-9, November.
    14. Alfredo R. Paloyo & Arndt Rüdiger Reichert & Holger Reinermann & Harald Tauchmann, 2011. "The Causal Link Between Financial Incentives and Weight Loss – An Evidence-based Survey of the Literature," Ruhr Economic Papers 0290, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
    15. Ana Pizarro-Carmona & Sofía Baena & Patricia Jiménez & Lucía Jiménez, 2021. "Protocol for the Implementation and Evaluation of a Mindfulness-Based Intervention for Caregivers of Children with Mental Disorders in a Clinical Setting," IJERPH, MDPI, vol. 18(20), pages 1-25, October.
    16. Duyen Thi Kim Nguyen & Karen K Leung & Lynn McIntyre & William A Ghali & Reg Sauve, 2013. "Does Integrated Management of Childhood Illness (IMCI) Training Improve the Skills of Health Workers? A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-1, June.
    17. Jori E Fleisher & Brianna J Sennott & Erica Myrick & Claire J Niemet & Monica Lee & Courtney M Whitelock & Maya Sanghvi & Yuanqing Liu & Bichun Ouyang & Deborah A Hall & Cynthia L Comella & Joshua Cho, 2020. "KICK OUT PD: Feasibility and quality of life in the pilot karate intervention to change kinematic outcomes in Parkinson’s Disease," PLOS ONE, Public Library of Science, vol. 15(9), pages 1-17, September.
    18. Shepherd, Keith D. & Shepherd, Gemma & Walsh, Markus G., 2015. "Land health surveillance and response: A framework for evidence-informed land management," Agricultural Systems, Elsevier, vol. 132(C), pages 93-106.
    19. Alfredo R. Paloyo & Arndt R. Reichert & Holger Reinermann & Harald Tauchmann, 2014. "The Causal Link Between Financial Incentives And Weight Loss: An Evidence-Based Survey Of The Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 28(3), pages 401-420, July.
    20. Deidda, Manuela & Geue, Claudia & Kreif, Noemi & Dundas, Ruth & McIntosh, Emma, 2019. "A framework for conducting economic evaluations alongside natural experiments," Social Science & Medicine, Elsevier, vol. 220(C), pages 353-361.
    21. Ramya Ramamoorthi & Daniel Gahreman & Timothy Skinner & Simon Moss, 2019. "The effect of yoga practice on glycemic control and other health parameters in the prediabetic state: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-21, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0000332. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.