IDEAS home Printed from https://ideas.repec.org/a/wly/camsys/v21y2025i4ne70082.html

The Extent of the Use of GRADE in Campbell Systematic Reviews: A Systematic Survey

Author

Listed:
  • Zhenjie Lian
  • Rui Wang
  • Xuping Song
  • Yunze Han
  • Qiyin Luo
  • Jing Tang
  • Xinye Guo
  • Yan Ma
  • Yue Hu
  • Xufei Luo
  • Yaolong Chen
  • Kehu Yang
  • Howard White
  • Vivian Welch

Abstract

Objective To conduct a systematic survey on the extent of the use of the grading of recommendations, assessment, development, and evaluations (GRADE) and other evidence grading systems in Campbell systematic reviews (SRs). Study Design and Settings Campbell SRs published before January 1st, 2024, that used evidence grading systems were included. General characteristics and details of a summary of findings (SoF) table and an evidence profile (EP) were independently extracted by two investigators. Results Among 234 SRs retrieved, 46 (19.66%) used evidence grading systems, all of which were GRADE. One used GRADE erroneously to rate the quality of individual studies rather than the body of evidence. The 45 SRs used GRADE to assess the certainty of a body of evidence and included 858 outcomes. Of them, the certainty in evidence for 32 were rated as high (3.73%), 170 were moderate (19.81%), 291 were low (33.92%), and 365 were very low (42.54%). Among the 1860 instances of downgrading and upgrading, the certainty in evidence was mostly downgraded for risk of bias (ROB) (1026, 55.16%) and imprecision (408, 21.94%). The large magnitude of effect (14, 0.75%) and plausible confounding (10, 0.54%) were the main upgraded factors. The proportions for higher certainty in evidence (including high and moderate) were larger in the international development (9.59%) and social welfare (7.55%) groups than in the other groups (1.37%). Conclusion Most Campbell SRs do not assess the GRADE certainty in evidence. Where evidence is evaluated, the quality of that evidence is mainly low or very low, and this is most commonly due to serious ROB or imprecision.

Suggested Citation

  • Zhenjie Lian & Rui Wang & Xuping Song & Yunze Han & Qiyin Luo & Jing Tang & Xinye Guo & Yan Ma & Yue Hu & Xufei Luo & Yaolong Chen & Kehu Yang & Howard White & Vivian Welch, 2025. "The Extent of the Use of GRADE in Campbell Systematic Reviews: A Systematic Survey," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(4), December.
    • Handle: RePEc:wly:camsys:v:21:y:2025:i:4:n:e70082
    DOI: 10.1002/cl2.70082
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/cl2.70082
    Download Restriction: no
    File URL: https://libkey.io/10.1002/cl2.70082?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Johnny Dyreborg & Hester Johnstone Lipscomb & Kent Nielsen & Marianne Törner & Kurt Rasmussen & Karen Bo Frydendall & Hans Bay & Ulrik Gensby & Elizabeth Bengtsen & Frank Guldenmund & Pete Kines, 2022. "Safety interventions for the prevention of accidents at work: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    2. Sarah L. Carthy & Colm B. Doody & Katie Cox & Denis O'Hora & Kiran M. Sarma, 2020. "Counter‐narratives for the prevention of violent radicalisation: A systematic review of targeted interventions," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(3), September.
    3. repec:wly:camsys:v:10:y:2014:i:1:p:1-56:a is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mia Doolan & Katie Cox & Kiran M. Sarma, 2025. "PROTOCOL: Understanding the Content, Context, and Impact of Far‐Right Extremist Propaganda Disseminated Online: A Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(4), December.
    2. Annette Meng & E. Sundstrup & Lars Louis Andersen, 2024. "The use of individual and collective selection, optimisation and compensation (SOC) strategies and their association with work ability among senior workers," European Journal of Ageing, Springer, vol. 21(1), pages 1-14, December.
    3. Pablo Madriaza & Ghayda Hassan & Sébastien Brouillette‐Alarie & Aoudou Njingouo Mounchingam & Loïc Durocher‐Corfa & Eugene Borokhovski & David Pickup & Sabrina Paillé, 2025. "Exposure to hate in online and traditional media: A systematic review and meta‐analysis of the impact of this exposure on individuals and communities," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(1), March.
    4. O'Connell, Michael, 2023. "Bridging the Gap Between Theory and Practice: An Analysis of the UK Government's Counter-Narrative Strategy in the Fight Against Online Extremist Narratives," OSF Preprints hdae5, Center for Open Science.
    5. Michael Wolfowicz & Badi Hasisi & David Weisburd, 2022. "What are the effects of different elements of media on radicalization outcomes? A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    6. Michael Wolfowicz & Badi Hasisi & David Weisburd, 2021. "PROTOCOL: What are the effects of different elements of media on radicalization outcomes? A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(1), March.
    7. Eva‐Marie Meemken & Olayinka Aremu & Anna Fabry & Celestina Heepen & Patrick Illien & Marie Kammer & Andrew Laitha, 2025. "Policy for Decent Work in Agriculture," Agricultural Economics, International Association of Agricultural Economists, vol. 56(3), pages 401-418, May.
    8. Isaac Onyeyirichukwu Chukwuma & Ifeanyi Leo Madu & Patrick Chukwunwike Chukwuma, 2025. "Workplace Safety: A Conceptual Review," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 9(14), pages 701-707, March.
    9. Opacin, Nerkez & Čehajić-Clancy, Sabina, 2025. "The potential of peacebuilding education interventions in societies facing the risk of extreme ideological shifts," International Journal of Educational Development, Elsevier, vol. 113(C).
    10. James Lewis & Sarah Marsden & Anna Stefaniak & James Hewitt, 2025. "PROTOCOL: Non‐criminal justice interventions for countering cognitive and behavioural radicalisation amongst children and adolescents: A systematic review of effectiveness and implementation," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(1), March.
    11. Peter Neyroud & Ajmal Aziz & Brett Kubicek, 2024. "Update on Campbell's Countering Violent Extremism programme," Campbell Systematic Reviews, John Wiley & Sons, vol. 20(2), June.
    12. Kiran M. Sarma & Sarah L. Carthy & Katie M. Cox, 2022. "PROTOCOL: Mental disorder, psychological problems and terrorist behaviour: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(2), June.
    13. Howard White, 2022. "Getting evidence into use: The experience of the Campbell Collaboration," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(1), March.
    14. Michelle Sydes & Lorelei Hine & Angela Higginson & James McEwan & Laura Dugan & Lorraine Mazerolle, 2023. "Criminal justice interventions for preventing radicalisation, violent extremism and terrorism: An evidence and gap map," Campbell Systematic Reviews, John Wiley & Sons, vol. 19(4), December.
    15. James Lewis & Sarah Marsden & James Hewitt & Chloe Squires & Anna Stefaniak, 2025. "Non‐Criminal Justice Interventions for Countering Cognitive and Behavioural Radicalisation Amongst Children and Adolescents: A Systematic Review of Effectiveness and Implementation," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(4), December.
    16. Sara Valdebenito & Manuel Eisner, 2025. "PROTOCOL: Interventions to Prevent Cognitive and Behavioural Violent Radicalisation: A Systematic Review and Multilevel Meta‐Analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(3), September.
    17. Felipe Pathé Duarte & João Pedro Ramos & Pedro Barbosa & Matteo Vergani & Cátia Moreira de Carvalho, 2025. "PROTOCOL: Effectiveness of Educational Programmes to Prevent and Counter Online Violent Extremist Propaganda in English, French, Spanish, Portuguese, German and Scandinavian Language Studies: A Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(2), June.
    18. repec:bjc:journl:v:12:y:2025:i:8:p:1728-1746 is not listed on IDEAS
    19. Kiran M. Sarma & Sarah L. Carthy & Katie M. Cox, 2022. "Mental disorder, psychological problems and terrorist behaviour: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 18(3), September.
    20. Ghayda Hassan & Sébastien Brouillette‐Alarie & Kurt Braddock & Sarah Carthy & Wynnpaul Varela & Pablo Madriaza & Paul Gill, 2025. "PROTOCOL: Government‐Led Communication Campaigns for Reducing Violent Extremism − A Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 21(2), June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:camsys:v:21:y:2025:i:4:n:e70082. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1891-1803 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.