IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/5zbh6.html
   My bibliography  Save this paper

Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods

Author

Listed:
  • Weber, Frank
  • Knapp, Guido
  • Glass, Anne
  • Kundt, Günther
  • Ickstadt, Katja

Abstract

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study is still lacking. Thus, we conduct such a simulation study for continuous and binary outcomes, focusing on the medical field for application. Based on the literature review and some new theoretical considerations, a practicable number of interval estimators is selected for this comparison: the classical normal-approximation interval using the DerSimonian-Laird heterogeneity estimator, the HKSJ interval using either the Paule-Mandel or the Sidik-Jonkman heterogeneity estimator, the Skovgaard higher-order profile likelihood interval, a parametric bootstrap interval, and a Bayesian interval using different priors. We evaluate the performance measures (coverage and interval length) at specific points in the parameter space, i.e. not averaging over a prior distribution. In this sense, our study is conducted from a frequentist point of view. We confirm the main finding of the literature review, the general recommendation of the HKSJ method (here with the Sidik-Jonkman heterogeneity estimator). For meta-analyses including only 2 studies, the high length of the HKSJ interval limits its practical usage. In this case, the Bayesian interval using a weakly informative prior for the heterogeneity may help. Our recommendations are illustrated using a real-world meta-analysis dealing with the efficacy of an intramyocardial bone marrow stem cell transplantation during coronary artery bypass grafting.

Suggested Citation

  • Weber, Frank & Knapp, Guido & Glass, Anne & Kundt, Günther & Ickstadt, Katja, 2020. "Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods," OSF Preprints 5zbh6, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:5zbh6
    DOI: 10.31219/osf.io/5zbh6
    as

    Download full text from publisher

    File URL: https://osf.io/download/5e6f7a6b0cd06c038a003096/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/5zbh6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Evangelos Kontopantelis & David A Springate & David Reeves, 2013. "A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses," PLOS ONE, Public Library of Science, vol. 8(7), pages 1-14, July.
    2. Kurex Sidik & Jeffrey N. Jonkman, 2005. "Simple heterogeneity variance estimation for meta‐analysis," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 54(2), pages 367-384, April.
    3. I. Kosmidis & A. Guolo & C. Varin, 2017. "‘Improving the accuracy of likelihood-based inference in meta-analysis and meta-regression’," Biometrika, Biometrika Trust, vol. 104(3), pages 751-751.
    4. Hajo Holzmann & Sebastian Vollmer, 2008. "A likelihood ratio test for bimodality in two-component mixtures with application to regional income distribution in the EU," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 92(1), pages 57-69, February.
    5. Friedrich, Thomas & Knapp, Guido, 2013. "Generalised interval estimation in the random effects meta regression model," Computational Statistics & Data Analysis, Elsevier, vol. 64(C), pages 165-179.
    6. Weber, Frank & Knapp, Guido & Ickstadt, Katja & Kundt, Günther & Glass, Anne, 2020. "Zero-cell corrections in random-effects meta-analyses," OSF Preprints qjh5f, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yu, Dalei & Ding, Chang & He, Na & Wang, Ruiwu & Zhou, Xiaohua & Shi, Lei, 2019. "Robust estimation and confidence interval in meta-regression models," Computational Statistics & Data Analysis, Elsevier, vol. 129(C), pages 93-118.
    2. Mathur, Maya B & VanderWeele, Tyler, 2017. "Sensitivity analysis for unmeasured confounding in meta-analyses," OSF Preprints jkhfg, Center for Open Science.
    3. Weber, Frank & Knapp, Guido & Ickstadt, Katja & Kundt, Günther & Glass, Anne, 2020. "Zero-cell corrections in random-effects meta-analyses," OSF Preprints qjh5f, Center for Open Science.
    4. Johannes Hönekopp & Audrey Helen Linden, 2022. "Heterogeneity estimates in a biased world," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-21, February.
    5. Sidik, Kurex & Jonkman, Jeffrey N., 2006. "Robust variance estimation for random effects meta-analysis," Computational Statistics & Data Analysis, Elsevier, vol. 50(12), pages 3681-3701, August.
    6. Filipe Manuel Clemente & Rodrigo Ramirez-Campillo & José Afonso & Hugo Sarmento & Thomas Rosemann & Beat Knechtle, 2021. "A Meta-Analytical Comparison of the Effects of Small-Sided Games vs. Running-Based High-Intensity Interval Training on Soccer Players’ Repeated-Sprint Ability," IJERPH, MDPI, vol. 18(5), pages 1-11, March.
    7. Xiaoli Hua & Lili Yu & Ruxu You & Yu Yang & Jing Liao & Dongsheng Chen & Lixiu Yu, 2016. "Association among Dietary Flavonoids, Flavonoid Subclasses and Ovarian Cancer Risk: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-15, March.
    8. H. Zakerzadeh & A. Jafari, 2015. "Inference on the parameters of two Weibull distributions based on record values," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(1), pages 25-40, March.
    9. Zhihua Ye & Xin Zhang & Yihuan Luo & Shikang Li & Lanshan Huang & Zuyun Li & Ping Li & Gang Chen, 2016. "Prognostic Values of Vimentin Expression and Its Clinicopathological Significance in Non-Small Cell Lung Cancer: A Meta-Analysis of Observational Studies with 4118 Cases," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-17, September.
    10. Yan Wang & Lan Huang & Li Zhang & Yi Qu & Dezhi Mu, 2017. "Iron Status in Attention-Deficit/Hyperactivity Disorder: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-14, January.
    11. Lara Carneiro & José Afonso & Rodrigo Ramirez-Campillo & Eugenia Murawska-Ciałowciz & Adilson Marques & Filipe Manuel Clemente, 2020. "The Effects of Exclusively Resistance Training-Based Supervised Programs in People with Depression: A Systematic Review and Meta-Analysis of Randomized Controlled Trials," IJERPH, MDPI, vol. 17(18), pages 1-22, September.
    12. Sebastian Vollmer & Hajo Holzmann & Florian Ketterer & Stephan Klasen, 2013. "Distribution dynamics of regional GDP per employee in unified Germany," Empirical Economics, Springer, vol. 44(2), pages 491-509, April.
    13. Ogundeji, Yewande Kofoworola & Bland, John Martin & Sheldon, Trevor Andrew, 2016. "The effectiveness of payment for performance in health care: A meta-analysis and exploration of variation in outcomes," Health Policy, Elsevier, vol. 120(10), pages 1141-1150.
    14. Min Wen & Bo Zhou & Yun-Hua Chen & Zhao-Lei Ma & Yun Gou & Chun-Lin Zhang & Wen-Feng Yu & Ling Jiao, 2017. "Serum uric acid levels in patients with Parkinson’s disease: A meta-analysis," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-13, March.
    15. Tseday Jemaneh Mekasha & Finn Tarp, 2019. "A Meta-Analysis of Aid Effectiveness: Revisiting the Evidence," Politics and Governance, Cogitatio Press, vol. 7(2), pages 5-28.
    16. Rodolfo Castro & Hugo Perazzo & Letícia Artilles Mello Mendonça de Araujo & Isabella Gonçalves Gutierres & Beatriz Grinsztejn & Valdiléa G Veloso, 2020. "Effectiveness of implementing a decentralized delivery of hepatitis C virus treatment with direct-acting antivirals: A systematic review with meta-analysis," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-14, February.
    17. Lanju Zhang & Zailong Wang & Li Wang & Lu Cui & Jeremy Sokolove & Ivan Chan, 2022. "A Simple Approach to Incorporating Historical Control Data in Clinical Trial Design and Analysis," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 216-236, July.
    18. Diandian Li & Yi Liu & Bo Wang, 2020. "Single versus bilateral lung transplantation in idiopathic pulmonary fibrosis: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-12, May.
    19. Martina Lori & Sarah Symnaczik & Paul Mäder & Gerlinde De Deyn & Andreas Gattinger, 2017. "Organic farming enhances soil microbial abundance and activity—A meta-analysis and meta-regression," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-25, July.
    20. Falonn Contreras-Osorio & Rodrigo Ramirez-Campillo & Enrique Cerda-Vega & Rodrigo Campos-Jara & Cristian Martínez-Salazar & Rodrigo Araneda & Daniela Ebner-Karestinos & Cristián Arellano-Roco & Christ, 2022. "Effects of Sport-Based Exercise Interventions on Executive Function in Older Adults: A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 19(19), pages 1-19, October.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:5zbh6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.