IDEAS home Printed from https://ideas.repec.org/a/wly/camsys/v17y2021i1ne1134.html
   My bibliography  Save this article

The methodological and reporting characteristics of Campbell reviews: A systematic review

Author

Listed:
  • Xiaoqin Wang
  • Vivian Welch
  • Meixuan Li
  • Liang Yao
  • Julia Littell
  • Huijuan Li
  • Nan Yang
  • Jianjian Wang
  • Larissa Shamseer
  • Yaolong Chen
  • Kehu Yang
  • Jeremy M. Grimshaw

Abstract

Background The Campbell Collaboration undertakes systematic reviews of the effects of social and economic policies (interventions) to help policymakers, practitioners, and the public to make well‐informed decisions about policy interventions. In 2010, the Cochrane Collaboration and the Campbell Collaboration developed a voluntary co‐registration policy under the rationale to make full use of the shared interests and diverse expertise from different review groups within these two organizations. In order to promote the methodological quality and transparency of Campbell intervention reviews, the Methodological Expectations of Campbell Collaboration Intervention Reviews (MECCIR) were introduced in 2014 to guide Campbell reviewers. However, there has not been a comprehensive review of the methodological quality and reporting characteristics of Campbell reviews. Objectives This review aimed to assess the methodological and reporting characteristics of Campbell intervention reviews and to compare the methodological quality and reporting completeness of Campbell reviews published before and after the implementation of MECCIR. A secondary aim was to compare the methodological quality and reporting completeness of reviews registered with Campbell only versus those co‐registered with Cochrane and Campbell. Search Methods We searched the Campbell Library to identify all the completed intervention reviews published between 1 January 2011 to 31 January 2018. Selection Criteria One researcher downloaded and screened all the records to exclude non‐intervention reviews based on reviews’ title and abstract. A second researcher checked the full text of all the excluded records to confirm the exclusion. In case of discrepancies, the two researchers jointly agreed on the final decision. Data Collection and Analysis We developed the abstraction form based on mandatory reporting items for methods, results, and discussion from the MECCIR reporting standards Version 1.1; and additional epidemiological characteristics identified in a similar study of systematic reviews in health. Additionally, we judged the methodological quality and completeness of reporting of each included review. For methodological quality, we used the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) instrument; for reporting completeness we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‐Analyses) checklist. We rated reporting as either complete/partial or not reported. We described characteristics of the included reviews with frequencies and percentages, and median with interquartile ranges (IQRs). We used Stata version 12.0 to conduct multiple linear regressions for continuous data and the ordered logistic regressions for ordered data to investigate associations between prespecified factors and both methodological quality and completeness of reporting. Main Results We included 96 Campbell reviews, 46 were published between January 2011 and September 2014 (pre‐MECCIR) and 50 between October 2014 and January 2018 (post‐MECCIR). Twenty‐two of 96 (23%) reviews were co‐registered with Cochrane. For overall methodological quality, 16 (17%) reviews were rated as high, 40 (42%) as moderate, 24 (25%) as low and 16 (17%) as critical low using AMSTAR 2. Reviews published after the release of MECCIR had better methodological quality ratings than those published before MECCIR (odds ratio [OR] =6.61, 95% confidence interval [CI] [2.86, 15.27], p

Suggested Citation

  • Xiaoqin Wang & Vivian Welch & Meixuan Li & Liang Yao & Julia Littell & Huijuan Li & Nan Yang & Jianjian Wang & Larissa Shamseer & Yaolong Chen & Kehu Yang & Jeremy M. Grimshaw, 2021. "The methodological and reporting characteristics of Campbell reviews: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    • Handle: RePEc:wly:camsys:v:17:y:2021:i:1:n:e1134
    DOI: 10.1002/cl2.1134
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/cl2.1134
    Download Restriction: no
    File URL: https://libkey.io/10.1002/cl2.1134?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David Moher & Jennifer Tetzlaff & Andrea C Tricco & Margaret Sampson & Douglas G Altman, 2007. "Epidemiology and Reporting Characteristics of Systematic Reviews," PLOS Medicine, Public Library of Science, vol. 4(3), pages 1-9, March.
    2. Xiaoqin Wang & Vivian Welch & Liang Yao & Julia H. Littell & Huijuan Li & Meixuan Li & Nan Yang & Jianjian Wang & Larissa Shamseer & Yaolong Chen & Kehu Yang & Jeremy M. Grimshaw, 2019. "PROTOCOL: The methodological and reporting characteristics of Campbell reviews: a methodological systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 15(1-2), June.
    3. David Moher & Alessandro Liberati & Jennifer Tetzlaff & Douglas G Altman & The PRISMA Group, 2009. "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-6, July.
    4. Alessandro Liberati & Douglas G Altman & Jennifer Tetzlaff & Cynthia Mulrow & Peter C Gøtzsche & John P A Ioannidis & Mike Clarke & P J Devereaux & Jos Kleijnen & David Moher, 2009. "The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-28, July.
    5. Vivian A. Welch, 2019. "Campbell systematic reviews takes next step to meeting FAIR principles," Campbell Systematic Reviews, John Wiley & Sons, vol. 15(1-2), June.
    6. Matthew J Page & Larissa Shamseer & Douglas G Altman & Jennifer Tetzlaff & Margaret Sampson & Andrea C Tricco & Ferrán Catalá-López & Lun Li & Emma K Reid & Rafael Sarkis-Onofre & David Moher, 2016. "Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study," PLOS Medicine, Public Library of Science, vol. 13(5), pages 1-30, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sarah Young & Alison Bethel & Ciara Keenan & Kate Ghezzi‐Kopel & Elizabeth Moreton & David Pickup & Zahra A. Premji & Morwenna Rogers & Bjørn C. A. Viinholt, 2021. "PROTOCOL: Searching and reporting in Campbell Collaboration systematic reviews: An assessment of current methods," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(4), December.
    2. Ariel M. Aloe & Ruth Garside, 2021. "Editorial: Types of methods research papers in the journal Campbell Systematic Reviews," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(2), June.
    3. Sarah Young & Heather MacDonald & Diana Louden & Ursula M. Ellis & Zahra Premji & Morwenna Rogers & Alison Bethel & David Pickup, 2024. "Searching and reporting in Campbell Collaboration systematic reviews: A systematic assessment of current methods," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(3), September.
    4. Liping Guo & Wenjie Zhou & Xin Xing & Zhipeng Wei & Minyan Yang & Mina Ma & Kehu Yang & Howard White, 2022. "PROTOCOL: Critical appraisal of methodological quality and reporting items of systematic reviews with meta‐analysis in evidence‐based social science in China: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(4), December.
    5. Vivian A. Welch, 2021. "Campbell Collaboration: Reflection on growth and cultivation from 2017 to 2021," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(4), December.
    6. Yanfei Li & Omar Dewidar & Xiaoqin Wang & Elizabeth Ghogomu & Arpana Wadhwani & Ke Guo & Mina Ma & Victoria Barbeau & Bei Pan & Leenah Abdelrazeq & Zijun Li & Amjad Alghamyan & Liping Guo & Fatima Jah, 2023. "Methodological quality of Campbell Systematic Reviews has improved over the past decade," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(4), December.
    7. Bei Pan & Long Ge & Zhipeng Wei & Liangying Hou & Honghao Lai & Kehu Yang, 2023. "PROTOCOL: Assessment of publication time in Campbell systematic reviews: A cross‐sectional survey," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(1), March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthew J Page & Joanne E McKenzie & Patrick M Bossuyt & Isabelle Boutron & Tammy C Hoffmann & Cynthia D Mulrow & Larissa Shamseer & Jennifer M Tetzlaff & Elie A Akl & Sue E Brennan & Roger Chou & Jul, 2021. "The PRISMA 2020 statement: An updated guideline for reporting systematic reviews," PLOS Medicine, Public Library of Science, vol. 18(3), pages 1-15, March.
    2. Neal R. Haddaway & Matthew J. Page & Chris C. Pritchard & Luke A. McGuinness, 2022. "PRISMA2020: An R package and Shiny app for producing PRISMA 2020‐compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    3. Giuseppe La Torre & Remigio Bova & Rosario Andrea Cocchiara & Cristina Sestili & Anna Tagliaferri & Simona Maggiacomo & Camilla Foschi & William Zomparelli & Maria Vittoria Manai & David Shaholli & Va, 2023. "What Are the Determinants of the Quality of Systematic Reviews in the International Journals of Occupational Medicine? A Methodological Study Review of Published Literature," IJERPH, MDPI, vol. 20(2), pages 1-12, January.
    4. Sarfraz Aslam & Atif Saleem & Teresa J. Kennedy & Tribhuwan Kumar & Khalida Parveen & Huma Akram & BaoHui Zhang, 2022. "Identifying the Research and Trends in STEM Education in Pakistan: A Systematic Literature Review," SAGE Open, , vol. 12(3), pages 21582440221, August.
    5. João Carlos Belloti & Aldo Okamura & Jordana Scheeren & Flávio Faloppa & Vinícius Ynoe de Moraes, 2019. "A systematic review of the quality of distal radius systematic reviews: Methodology and reporting assessment," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-12, January.
    6. Lucy Turner & James Galipeau & Chantelle Garritty & Eric Manheimer & L Susan Wieland & Fatemeh Yazdi & David Moher, 2013. "An Evaluation of Epidemiological and Reporting Characteristics of Complementary and Alternative Medicine (CAM) Systematic Reviews (SRs)," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    7. Abeer Elshater & Hisham Abusaada, 2022. "Developing Process for Selecting Research Techniques in Urban Planning and Urban Design with a PRISMA-Compliant Review," Social Sciences, MDPI, vol. 11(10), pages 1-17, October.
    8. Alessandro Concari & Gerjo Kok & Pim Martens, 2020. "A Systematic Literature Review of Concepts and Factors Related to Pro-Environmental Consumer Behaviour in Relation to Waste Management Through an Interdisciplinary Approach," Sustainability, MDPI, vol. 12(11), pages 1-50, May.
    9. Nikolaos Pandis & Padhraig S Fleming & Helen Worthington & Kerry Dwan & Georgia Salanti, 2015. "Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-10, September.
    10. Frank Peinemann & Ulrich Grouven & Nicolaus Kröger & Carmen Bartel & Max H Pittler & Stefan Lange, 2011. "First-Line Matched Related Donor Hematopoietic Stem Cell Transplantation Compared to Immunosuppressive Therapy in Acquired Severe Aplastic Anemia," PLOS ONE, Public Library of Science, vol. 6(4), pages 1-16, April.
    11. Jonathan Kingsley & Aisling Bailey & Nooshin Torabi & Pauline Zardo & Suzanne Mavoa & Tonia Gray & Danielle Tracey & Philip Pettitt & Nicholas Zajac & Emily Foenander, 2019. "A Systematic Review Protocol Investigating Community Gardening Impact Measures," IJERPH, MDPI, vol. 16(18), pages 1-12, September.
    12. S. Rajsic & H. Gothe & H. H. Borba & G. Sroczynski & J. Vujicic & T. Toell & Uwe Siebert, 2019. "Economic burden of stroke: a systematic review on post-stroke care," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(1), pages 107-134, February.
    13. Sibonokuhle Ndlovu, 2023. "Preparedness and Response to COVID-19 Disruptions and Learning Challenges for Students with Disabilities in South Africa: A Systematic Review," Sustainability, MDPI, vol. 15(2), pages 1-17, January.
    14. Abbas Mardani & Dalia Streimikiene & Edmundas Kazimieras Zavadskas & Fausto Cavallaro & Mehrbakhsh Nilashi & Ahmad Jusoh & Habib Zare, 2017. "Application of Structural Equation Modeling (SEM) to Solve Environmental Sustainability Problems: A Comprehensive Review and Meta-Analysis," Sustainability, MDPI, vol. 9(10), pages 1-65, October.
    15. Luísa Bandeira Lopes & João Albernaz Neves & João Botelho & Vanessa Machado & José João Mendes, 2021. "Regenerative Endodontic Procedures: An Umbrella Review," IJERPH, MDPI, vol. 18(2), pages 1-17, January.
    16. Gaspar, Rui & Yan, Zheng & Domingos, Samuel, 2019. "Extreme natural and man-made events and human adaptive responses mediated by information and communication technologies' use: A systematic literature review," Technological Forecasting and Social Change, Elsevier, vol. 145(C), pages 125-135.
    17. Nicholas A. Kirk & Nicholas A. Cradock-Henry, 2022. "Land Management Change as Adaptation to Climate and Other Stressors: A Systematic Review of Decision Contexts Using Values-Rules-Knowledge," Land, MDPI, vol. 11(6), pages 1-23, May.
    18. Timothy Noblet & John Marriott & Emma Graham-Clarke & Debra Shirley & Alison Rushton, 2018. "Clinical and cost-effectiveness of non-medical prescribing: A systematic review of randomised controlled trials," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    19. Abbas Mardani & Mehrbakhsh Nilashi & Jurgita Antucheviciene & Madjid Tavana & Romualdas Bausys & Othman Ibrahim, 2017. "Recent Fuzzy Generalisations of Rough Sets Theory: A Systematic Review and Methodological Critique of the Literature," Complexity, Hindawi, vol. 2017, pages 1-33, October.
    20. Wen-Hsiu Yeh & Ya-Ju Ju & Yu-Ting Liu & Ting-Yi Wang, 2022. "Systematic Review and Meta-Analysis on the Effects of Neurofeedback Training of Theta Activity on Working Memory and Episodic Memory in Healthy Population," IJERPH, MDPI, vol. 19(17), pages 1-14, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:camsys:v:17:y:2021:i:1:n:e1134. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1891-1803 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.