IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v9y2009i2p197-210.html
   My bibliography  Save this article

Updated tests for small-study effects in meta-analyses

Author

Listed:
  • Roger M. Harbord

    (University of Bristol)

  • Ross J. Harris

    (Health Protection Agency, UK)

  • Jonathan A. C. Sterne

    (University of Bristol)

Abstract

This article describes an updated version of the metabias command, which provides statistical tests for funnel plot asymmetry. In addition to the previously implemented tests, metabias implements two new tests that are recom- mended in the recently updated Cochrane Handbook for Systematic Reviews of Interventions (Higgins and Green 2008). The first new test, proposed by Harbord, Egger, and Sterne (2006, Statistics in Medicine 25: 3443–3457), is a modified ver- sion of the commonly used test proposed by Egger et al. (1997, British Medical Journal 315: 629–634). It regresses Z/√V against √V , where Z is the efficient score and V is Fisher’s information (the variance of Z under the null hypothesis). The second new test is Peters’ test, which is based on a weighted linear regression of the intervention effect estimate on the reciprocal of the sample size. Both of these tests maintain better control of the false-positive rate than the test proposed by Egger at al., while retaining similar power. Copyright 2009 by StataCorp LP.

Suggested Citation

  • Roger M. Harbord & Ross J. Harris & Jonathan A. C. Sterne, 2009. "Updated tests for small-study effects in meta-analyses," Stata Journal, StataCorp LP, vol. 9(2), pages 197-210, June.
    • Handle: RePEc:tsj:stataj:v:9:y:2009:i:2:p:197-210
    as

    Download full text from publisher

    File URL: http://www.stata-journal.com/article.html?article=sbe19_6
    File Function: link to article download
    Download Restriction: no
    File URL: http://www.stata-journal.com/software/sj9-2/sbe19_6/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jonathan A.C. Sterne & Roger M. Harbord, 2004. "Funnel plots in meta-analysis," Stata Journal, StataCorp LP, vol. 4(2), pages 127-141, June.
    2. Thomas J. Steichen & Matthias Egger & Jonathan Sterne, 1999. "Tests for publication bias in meta-analysis," Stata Technical Bulletin, StataCorp LP, vol. 8(44).
    3. Colin B. Begg & Jesse A. Berlin, 1988. "Publication Bias: A Problem in Interpreting Medical Data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 151(3), pages 419-445, May.
    4. Tom M. Palmer & Jaime L. Peters & Alex J. Sutton & Santiago G. Moreno, 2008. "Contour-enhanced funnel plots for meta-analysis," Stata Journal, StataCorp LP, vol. 8(2), pages 242-254, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuan Yuan Hu & Hua Yuan & Guang Bing Jiang & Ning Chen & Li Wen & Wei Dong Leng & Xian Tao Zeng & Yu Ming Niu, 2012. "Associations between XPD Asp312Asn Polymorphism and Risk of Head and Neck Cancer: A Meta-Analysis Based on 7,122 Subjects," PLOS ONE, Public Library of Science, vol. 7(4), pages 1-6, April.
    2. Marta Liliana Musskopf & Amanda Finger Stadler & Ulf ME Wikesjö & Cristiano Susin, 2022. "The minipig intraoral dental implant model: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-17, February.
    3. Nick Freemantle, 2008. "Biostatistical aspects for the use of evidence based medicine in health technology assessment," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 9(1), pages 31-43, November.
    4. Mohammad O Dahl, 2020. "Brucellosis in food-producing animals in Mosul, Iraq: A systematic review and meta-analysis," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-16, July.
    5. Allison Milner & Andrew Page & Anthony D LaMontagne, 2013. "Long-Term Unemployment and Suicide: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-6, January.
    6. Jonathan Stokes & Maria Panagioti & Rahul Alam & Kath Checkland & Sudeh Cheraghi-Sohi & Peter Bower, 2015. "Effectiveness of Case Management for 'At Risk' Patients in Primary Care: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-42, July.
    7. Sanghyun Hong & W. Robert Reed, 2020. "Using Monte Carlo Experiments to Select Meta-Analytic Estimators," Working Papers in Economics 20/10, University of Canterbury, Department of Economics and Finance.
    8. Catarina Gonçalves & Armando Raimundo & Ana Abreu & Jorge Bravo, 2021. "Exercise Intensity in Patients with Cardiovascular Diseases: Systematic Review with Meta-Analysis," IJERPH, MDPI, vol. 18(7), pages 1-19, March.
    9. Qiaoning Yue & Yue Ma & Yirong Teng & Yun Zhu & Hao Liu & Shuanglan Xu & Jie Liu & Jianping Liu & Xiguang Zhang & Zhaowei Teng, 2020. "An updated analysis of opioids increasing the risk of fractures," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-12, April.
    10. Qiang Huo & Ning Zhang & Qifeng Yang, 2012. "Epstein-Barr Virus Infection and Sporadic Breast Cancer Risk: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-8, February.
    11. Qurat-ul-Ann, Abre-Rehmat & Mirza, Faisal Mehmood, 2020. "Meta-analysis of empirical evidence on energy poverty: The case of developing economies," Energy Policy, Elsevier, vol. 141(C).
    12. Sandra Jo Wilson & Emily E. Tanner‐Smith & Mark W. Lipsey & Katarzyna Steinka‐Fry & Jan Morrison, 2011. "Dropout Prevention and Intervention Programs: Effects on School Completion and Dropout among School‐aged Children and Youth," Campbell Systematic Reviews, John Wiley & Sons, vol. 7(1), pages 1-61.
    13. Tseday Jemaneh Mekasha & Finn Tarp, 2013. "Aid and Growth: What Meta-Analysis Reveals," Journal of Development Studies, Taylor & Francis Journals, vol. 49(4), pages 564-583, April.
    14. Meng-Bo Hu & Sheng-Hua Liu & Hao-Wen Jiang & Pei-De Bai & Qiang Ding, 2014. "Obesity Affects the Biopsy-Mediated Detection of Prostate Cancer, Particularly High-Grade Prostate Cancer: A Dose-Response Meta-Analysis of 29,464 Patients," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-10, September.
    15. Lin Zhao & Aihua Gu & Guixiang Ji & Peng Zou & Peng Zhao & Ailin Lu, 2012. "The Association between ATM IVS 22-77 T>C and Cancer Risk: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 7(1), pages 1-7, January.
    16. Tseday Jemaneh Mekasha & Finn Tarp, 2013. "Aid and Growth: What Meta-Analysis Reveals," Journal of Development Studies, Taylor & Francis Journals, vol. 49(4), pages 564-583, April.
    17. Sevigny, Eric L. & Fuleihan, Brian K. & Ferdik, Frank V., 2013. "Do drug courts reduce the use of incarceration?: A meta-analysis," Journal of Criminal Justice, Elsevier, vol. 41(6), pages 416-425.
    18. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    19. Jingyao Fan & Yiqing Song & Yuyao Wang & Rutai Hui & Weili Zhang, 2012. "Dietary Glycemic Index, Glycemic Load, and Risk of Coronary Heart Disease, Stroke, and Stroke Mortality: A Systematic Review with Meta-Analysis," PLOS ONE, Public Library of Science, vol. 7(12), pages 1-12, December.
    20. Shan Liu & Qing Li & Ying Zhang & Qiushuang Li & Baodong Ye & Dijiong Wu & Li Wu & Hanti Lu & Conghua Ji, 2016. "Association of Human Leukocyte Antigen DRB1*15 and DRB1*15:01 Polymorphisms with Response to Immunosuppressive Therapy in Patients with Aplastic Anemia: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 11(9), pages 1-15, September.
    21. Yi Gong & Leida Zhang & Ping Bie & Huaizhi Wang, 2013. "Roles of ApoB-100 Gene Polymorphisms and the Risks of Gallstones and Gallbladder Cancer: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 8(4), pages 1-9, April.
    22. James B Kirkbride & Antonia Errazuriz & Tim J Croudace & Craig Morgan & Daniel Jackson & Jane Boydell & Robin M Murray & Peter B Jones, 2012. "Incidence of Schizophrenia and Other Psychoses in England, 1950–2009: A Systematic Review and Meta-Analyses," PLOS ONE, Public Library of Science, vol. 7(3), pages 1-1, March.
    23. Ngamaba, Kayonda Hubert & Armitage, Christopher & Panagioti, Maria & Hodkinson, Alexander, 2020. "How closely related are financial satisfaction and subjective well-being? Systematic review and meta-analysis," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    24. Maamar Sebri, 2014. "A meta-analysis of residential water demand studies," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 16(3), pages 499-520, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael J. Crowther & Dean Langan & Alex J. Sutton, 2012. "Graphical augmentations to the funnel plot to assess the impact of a new study on an existing meta-analysis," Stata Journal, StataCorp LP, vol. 12(4), pages 605-622, December.
    2. Nino Fonseca & Marcelino Sánchez-Rivero, 2020. "Significance bias in the tourism-led growth literature," Tourism Economics, , vol. 26(1), pages 137-154, February.
    3. Wamisho, Kassu, 2013. "A Meta Regression Analysis of Soil Organic Carbon Sequestration in Corn Belt States: Implication for Cellulosic Biofuel Production," 2013 Annual Meeting, August 4-6, 2013, Washington, D.C. 149741, Agricultural and Applied Economics Association.
    4. Pérez-Calero, Leticia & Hurtado-González, José Manuel & López-Iturriaga, Félix J., 2019. "Do the institutional environment and types of owners influence the relationship between ownership concentration and board of director independence? An international meta-analysis," International Review of Financial Analysis, Elsevier, vol. 61(C), pages 233-244.
    5. Tom M. Palmer & Jaime L. Peters & Alex J. Sutton & Santiago G. Moreno, 2008. "Contour-enhanced funnel plots for meta-analysis," Stata Journal, StataCorp LP, vol. 8(2), pages 242-254, June.
    6. Ma, Chunbo & Rogers, Abbie A. & Kragt, Marit E. & Zhang, Fan & Polyakov, Maksym & Gibson, Fiona & Chalak, Morteza & Pandit, Ram & Tapsuwan, Sorada, 2015. "Consumers’ willingness to pay for renewable energy: A meta-regression analysis," Resource and Energy Economics, Elsevier, vol. 42(C), pages 93-109.
    7. Stanley, T.D. & Doucouliagos, Chris & Jarrell, Stephen B., 2008. "Meta-regression analysis as the socio-economics of economics research," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(1), pages 276-292, February.
    8. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
    9. Gundula Krack, 2019. "How to make value-based health insurance designs more effective? A systematic review and meta-analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 841-856, August.
    10. Solomiia Brychka & Denys Klynovskyi & Dmytro Krukovets & Artem Oharkov, 2019. "Meta-Analysis: Meta-Analysis: Effect of FX interventions on the exchange rate," Modern Economic Studies, Kyiv School of Economics, vol. 2(1), pages 24-44.
    11. Chatfield, Chris, 1995. "Positive or negative?," International Journal of Forecasting, Elsevier, vol. 11(4), pages 501-502, December.
    12. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    13. Mattia Filomena & Matteo Picchio, 2023. "Retirement and health outcomes in a meta‐analytical framework," Journal of Economic Surveys, Wiley Blackwell, vol. 37(4), pages 1120-1155, September.
    14. Auspurg Katrin & Hinz Thomas, 2011. "What Fuels Publication Bias?: Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 636-660, October.
    15. Amal F. Alshammary & Khalid Khalaf Alharbi & Naif Jameel Alshehri & Vishal Vennu & Imran Ali Khan, 2021. "Metabolic Syndrome and Coronary Artery Disease Risk: A Meta-Analysis of Observational Studies," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    16. Ugur, Mehmet & Awaworyi, Sefa & Solomon, Edna, 2016. "Technological innovation and employment in derived labour demand models: A hierarchical meta-regression analysis," MPRA Paper 73557, University Library of Munich, Germany.
    17. Rosenberger, Randall S. & Stanley, Tom D., 2006. "Measurement, generalization, and publication: Sources of error in benefit transfers and their management," Ecological Economics, Elsevier, vol. 60(2), pages 372-378, December.
    18. Ugur, Mehmet & Trushin, Eshref & Solomon, Edna & Guidi, Francesco, 2016. "R&D and productivity in OECD firms and industries: A hierarchical meta-regression analysis," Research Policy, Elsevier, vol. 45(10), pages 2069-2086.
    19. Ruxiu Tie & Tiansong Zhang & Huarui Fu & Limengmeng Wang & Yebo Wang & Ying He & Binsheng Wang & Ni Zhu & Shan Fu & Xiaoyu Lai & Jimin Shi & He Huang, 2014. "Association between DNMT3A Mutations and Prognosis of Adults with De Novo Acute Myeloid Leukemia: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    20. Da-Jung Ha & Jung-Hyun Park & Su-Eun Jung & Boram Lee & Myo-Sung Kim & Kyo-Lin Sim & Yung-Hyun Choi & Chan-Young Kwon, 2021. "The Experience of Emotional Labor and Its Related Factors among Nurses in General Hospital Settings in Republic of Korea: A Systematic Review and Meta-Analysis," Sustainability, MDPI, vol. 13(21), pages 1-21, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:9:y:2009:i:2:p:197-210. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum or Lisa Gilmore (email available below). General contact details of provider: http://www.stata-journal.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.