IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v173y2010i3p575-591.html
   My bibliography  Save this article

Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity

Author

Listed:
  • Jaime L. Peters
  • Alex J. Sutton
  • David R. Jones
  • Keith R. Abrams
  • Lesley Rushton
  • Santiago G. Moreno

Abstract

Summary. Between‐study heterogeneity and publication bias are common features of a meta‐analysis that can be present simultaneously. When both are suspected, consideration must be made of each in the assessment of the other. We consider extended funnel plot tests for detecting publication bias, and selection modelling and trim‐and‐fill methods to adjust for publication bias in the presence of between‐study heterogeneity. These methods are applied to two example data sets. Results indicate that ignoring between‐study heterogeneity when assessing publication bias can be misleading, but that methods to test or adjust for publication bias in the presence of heterogeneity may not be powerful when the meta‐analysis is not large. It is therefore unrealistic to expect to disentangle the effects of publication bias and heterogeneity reliably in all except the largest meta‐analyses.

Suggested Citation

  • Jaime L. Peters & Alex J. Sutton & David R. Jones & Keith R. Abrams & Lesley Rushton & Santiago G. Moreno, 2010. "Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 575-591, July.
    • Handle: RePEc:bla:jorssa:v:173:y:2010:i:3:p:575-591
    DOI: 10.1111/j.1467-985X.2009.00629.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-985X.2009.00629.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-985X.2009.00629.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    2. J. Copas, 1999. "What works?: selectivity models and meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 162(1), pages 95-109.
    3. Larry V. Hedges & Jack L. Vevea, 1996. "Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model," Journal of Educational and Behavioral Statistics, , vol. 21(4), pages 299-332, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:osf:osfxxx:gv25c_v1 is not listed on IDEAS
    2. Sergio Nolazco & Kaspar Delhey & Shinichi Nakagawa & Anne Peters, 2022. "Ornaments are equally informative in male and female birds," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    3. Celbis, Mehmet Güney & Nijkamp, Peter & Poot, Jacques, 2013. "How big is the impact of infrastructure on trade? Evidence from meta-analysis," MERIT Working Papers 2013-032, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    4. Dan Jackson, 2018. "Discussion on Quantifying publication bias in meta‐analysis," Biometrics, The International Biometric Society, vol. 74(3), pages 795-796, September.
    5. Polák, Petr, 2017. "The productivity paradox: A meta-analysis," Information Economics and Policy, Elsevier, vol. 38(C), pages 38-54.
    6. Augusteijn, Hilde & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2017. "The Effect of Publication Bias on the Assessment of Heterogeneity," OSF Preprints gv25c, Center for Open Science.
    7. Anna Chaimani & Julian P T Higgins & Dimitris Mavridis & Panagiota Spyridonos & Georgia Salanti, 2013. "Graphical Tools for Network Meta-Analysis in STATA," PLOS ONE, Public Library of Science, vol. 8(10), pages 1-12, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Irsova, Zuzana & Bom, Pedro Ricardo Duarte & Havranek, Tomas & Rachinger, Heiko, 2023. "Spurious Precision in Meta-Analysis," MetaArXiv 3qp2w, Center for Open Science.
    2. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    3. repec:osf:metaar:3qp2w_v2 is not listed on IDEAS
    4. repec:osf:metaar:3qp2w_v1 is not listed on IDEAS
    5. Ao Huang & Kosuke Morikawa & Tim Friede & Satoshi Hattori, 2023. "Adjusting for publication bias in meta‐analysis via inverse probability weighting using clinical trial registries," Biometrics, The International Biometric Society, vol. 79(3), pages 2089-2102, September.
    6. Sanghyun Hong & W. Robert Reed, 2020. "Using Monte Carlo Experiments to Select Meta-Analytic Estimators," Working Papers in Economics 20/10, University of Canterbury, Department of Economics and Finance.
    7. van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2018. "P-uniform," MetaArXiv zqjr9, Center for Open Science.
    8. Mengke Li & Yukun Liu & Pengfei Li & Jing Qin, 2022. "Empirical likelihood meta-analysis with publication bias correction under Copas-like selection model," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 74(1), pages 93-112, February.
    9. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    10. Yucheon Kim & Songyi Lee, 2023. "A Systematic Review and Meta-Analysis of the Effectiveness of Non-Face-to-Face Coaching," Sustainability, MDPI, vol. 15(12), pages 1-15, June.
    11. repec:plo:pone00:0171654 is not listed on IDEAS
    12. Zuzana Irsova & Hristos Doucouliagos & Tomas Havranek & T. D. Stanley, 2024. "Meta‐analysis of social science research: A practitioner's guide," Journal of Economic Surveys, Wiley Blackwell, vol. 38(5), pages 1547-1566, December.
    13. Qian Li & Yan Chen & Shikun Sun & Muyuan Zhu & Jing Xue & Zihan Gao & Jinfeng Zhao & Yihe Tang, 2022. "Research on Crop Irrigation Schedules Under Deficit Irrigation—A Meta-analysis," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 36(12), pages 4799-4817, September.
    14. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    15. Damiano Pizzol & Mike Trott & Igor Grabovac & Mario Antunes & Anna Claudia Colangelo & Simona Ippoliti & Cristian Petre Ilie & Anne Carrie & Nicola Veronese & Lee Smith, 2021. "Laparoscopy in Low-Income Countries: 10-Year Experience and Systematic Literature Review," IJERPH, MDPI, vol. 18(11), pages 1-11, May.
    16. Ferrero, Marta & Vadillo, Miguel A. & León, Samuel P., 2021. "A valid evaluation of the theory of multiple intelligences is not yet possible: Problems of methodological quality for intervention studies," Intelligence, Elsevier, vol. 88(C).
    17. Wolfgang Goymann & John C. Wingfield, 2014. "Male-to-female testosterone ratios, dimorphism, and life history—what does it really tell us?," Behavioral Ecology, International Society for Behavioral Ecology, vol. 25(4), pages 685-699.
    18. repec:plo:pone00:0204887 is not listed on IDEAS
    19. Alderotti, Giammarco & Rapallini, Chiara & Traverso, Silvio, 2023. "The Big Five personality traits and earnings: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 94(C).
    20. Zhigang Chen & Jiye Zhu & Chaoyuan Huang & Fang Lian & Guobin Wu & Yinnong Zhao, 2015. "The Association between Three Cyclooxygenase-2 Polymorphisms and Hepatocellular Carcinoma Risk: A Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(3), pages 1-12, March.
    21. Ünal, Zehra E. & Kartal, Gamze & Ulusoy, Serra & Ala, Aslı M. & Yilmaz, Munube & Geary, David C., 2023. "Relative contributions of g and basic domain-specific mathematics skills to complex mathematics competencies," Intelligence, Elsevier, vol. 101(C).
    22. Jo Smith & Xiao Qi Ang & Emma L. Giles & Gemma Traviss-Turner, 2023. "Emotional Eating Interventions for Adults Living with Overweight or Obesity: A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 20(3), pages 1-22, February.
    23. Daniele Zago & Maria Eugênia Andrighetto Canozzi & Júlio Otávio Jardim Barcellos, 2020. "Pregnant beef cow’s nutrition and its effects on postnatal weight and carcass quality of their progeny," PLOS ONE, Public Library of Science, vol. 15(8), pages 1-20, August.
    24. Jenny Jing Wen Liu & Natalie Ein & Julia Gervasio & Mira Battaion & Kenneth Fung, 2022. "The Pursuit of Resilience: A Meta-Analysis and Systematic Review of Resilience-Promoting Interventions," Journal of Happiness Studies, Springer, vol. 23(4), pages 1771-1791, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:173:y:2010:i:3:p:575-591. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.