IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0048894.html
   My bibliography  Save this article

A Comparison of Statistical Methods for Identifying Out-of-Date Systematic Reviews

Author

Listed:
  • Porjai Pattanittum
  • Malinee Laopaiboon
  • David Moher
  • Pisake Lumbiganon
  • Chetta Ngamjarus

Abstract

Background: Systematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful. This study aimed to compare five statistical methods to identify out-of-date SRs. Methods: A retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and “3-years previous” meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods was used, with random effects analyses throughout the study. Results: Eighty reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method. The overall agreement among the three discriminating statistical methods was slight (Kappa = 0.14; 95% CI 0.05 to 0.23). The recursive cumulative meta-analysis, Ottawa, and Barrowman methods were practical according to the study criteria. Conclusion: Our study shows that three practical statistical methods could be applied to examine the need to update SRs.

Suggested Citation

  • Porjai Pattanittum & Malinee Laopaiboon & David Moher & Pisake Lumbiganon & Chetta Ngamjarus, 2012. "A Comparison of Statistical Methods for Identifying Out-of-Date Systematic Reviews," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-6, November.
    • Handle: RePEc:plo:pone00:0048894
    DOI: 10.1371/journal.pone.0048894
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0048894
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0048894&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0048894?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David Moher & Jennifer Tetzlaff & Andrea C Tricco & Margaret Sampson & Douglas G Altman, 2007. "Epidemiology and Reporting Characteristics of Systematic Reviews," PLOS Medicine, Public Library of Science, vol. 4(3), pages 1-9, March.
    2. Alessandro Liberati & Douglas G Altman & Jennifer Tetzlaff & Cynthia Mulrow & Peter C Gøtzsche & John P A Ioannidis & Mike Clarke & P J Devereaux & Jos Kleijnen & David Moher, 2009. "The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-28, July.
    3. Hilda Bastian & Paul Glasziou & Iain Chalmers, 2010. "Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?," PLOS Medicine, Public Library of Science, vol. 7(9), pages 1-6, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucy Turner & James Galipeau & Chantelle Garritty & Eric Manheimer & L Susan Wieland & Fatemeh Yazdi & David Moher, 2013. "An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    2. Arjun K Reddy & Jared T Scott & Grayson R Norris & Chip Moore & Jake X Checketts & Griffin K Hughes & Travis Small & Mark M Calder & Brent L Norris, 2023. "Cemented vs Uncemented hemiarthroplasties for femoral neck fractures: An overlapping systematic review and evidence appraisal," PLOS ONE, Public Library of Science, vol. 18(2), pages 1-18, February.
    3. Giuseppe La Torre & Remigio Bova & Rosario Andrea Cocchiara & Cristina Sestili & Anna Tagliaferri & Simona Maggiacomo & Camilla Foschi & William Zomparelli & Maria Vittoria Manai & David Shaholli & Va, 2023. "What Are the Determinants of the Quality of Systematic Reviews in the International Journals of Occupational Medicine? A Methodological Study Review of Published Literature," IJERPH, MDPI, vol. 20(2), pages 1-12, January.
    4. Blake M. Louscher & Veerasathpurush Allareddy & Satheesh Elangovan, 2019. "Predictors of Citations of Systematic Reviews in Oral Implantology: A Cross-Sectional Bibliometric Analysis," SAGE Open, , vol. 9(1), pages 21582440198, March.
    5. João Carlos Belloti & Aldo Okamura & Jordana Scheeren & Flávio Faloppa & Vinícius Ynoe de Moraes, 2019. "A systematic review of the quality of distal radius systematic reviews: Methodology and reporting assessment," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-12, January.
    6. Hansen, Henrik & Trifkovic, Neda, 2013. "Systematic Reviews: Questions, Methods and Usage," MPRA Paper 47993, University Library of Munich, Germany.
    7. Jamie J Kirkham & Doug G Altman & Paula R Williamson, 2010. "Bias Due to Changes in Specified Outcomes during the Systematic Review Process," PLOS ONE, Public Library of Science, vol. 5(3), pages 1-5, March.
    8. Matthew J Page & Joanne E McKenzie & Patrick M Bossuyt & Isabelle Boutron & Tammy C Hoffmann & Cynthia D Mulrow & Larissa Shamseer & Jennifer M Tetzlaff & Elie A Akl & Sue E Brennan & Roger Chou & Jul, 2021. "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," PLOS Medicine, Public Library of Science, vol. 18(3), pages 1-15, March.
    9. repec:plo:pone00:0125931 is not listed on IDEAS
    10. Xiaoqin Wang & Vivian Welch & Meixuan Li & Liang Yao & Julia Littell & Huijuan Li & Nan Yang & Jianjian Wang & Larissa Shamseer & Yaolong Chen & Kehu Yang & Jeremy M. Grimshaw, 2021. "The methodological and reporting characteristics of Campbell reviews: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    11. repec:plo:pone00:0023051 is not listed on IDEAS
    12. Laura Sheble, 2017. "Macro‐level diffusion of a methodological knowledge innovation: Research synthesis methods, 1972–2011," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2693-2708, December.
    13. Abeer Elshater & Hisham Abusaada, 2022. "Developing Process for Selecting Research Techniques in Urban Planning and Urban Design with a PRISMA-Compliant Review," Social Sciences, MDPI, vol. 11(10), pages 1-17, October.
    14. Milena Dalton & Benjamin Sanderson & Leanne J Robinson & Caroline S E Homer & William Pomat & Margie Danchin & Stefanie Vaccher, 2023. "Impact of COVID-19 on routine childhood immunisations in low- and middle-income countries: A scoping review," PLOS Global Public Health, Public Library of Science, vol. 3(8), pages 1-17, August.
    15. Tripathy, Prajukta & Jena, Pabitra Kumar & Mishra, Bikash Ranjan, 2024. "Systematic literature review and bibliometric analysis of energy efficiency," Renewable and Sustainable Energy Reviews, Elsevier, vol. 200(C).
    16. Elizabeth T Cafiero-Fonseca & Andrew Stawasz & Sydney T Johnson & Reiko Sato & David E Bloom, 2017. "The full benefits of adult pneumococcal vaccination: A systematic review," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-23, October.
    17. Ludoviko Zirimenya & Fatima Mahmud-Ajeigbe & Ruth McQuillan & You Li, 2020. "A systematic review and meta-analysis to assess the association between urogenital schistosomiasis and HIV/AIDS infection," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 14(6), pages 1-13, June.
    18. Desalegne Amare & Fentie Ambaw Getahun & Endalkachew Worku Mengesha & Getenet Dessie & Melashu Balew Shiferaw & Tegenaw Asemamaw Dires & Kefyalew Addis Alene, 2023. "Effectiveness of healthcare workers and volunteers training on improving tuberculosis case detection: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 18(3), pages 1-13, March.
    19. Trang Nguyen & Sara Holton & Thach Tran & Jane Fisher, 2019. "Informal mental health interventions for people with severe mental illness in low and lower middle-income countries: A systematic review of effectiveness," International Journal of Social Psychiatry, , vol. 65(3), pages 194-206, May.
    20. Natalya Ivanova & Ekaterina Zolotova, 2023. "Landolt Indicator Values in Modern Research: A Review," Sustainability, MDPI, vol. 15(12), pages 1-22, June.
    21. Neal R. Haddaway & Matthew J. Page & Chris C. Pritchard & Luke A. McGuinness, 2022. "PRISMA2020: An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    22. Su Keng Tan & Wai Keung Leung & Alexander Tin Hong Tang & Roger A Zwahlen, 2017. "Effects of mandibular setback with or without maxillary advancement osteotomies on pharyngeal airways: An overview of systematic reviews," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-20, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0048894. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.