IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/avkgj.html
   My bibliography  Save this paper

Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias

Author

Listed:
  • Augusteijn, Hilde Elisabeth Maria

    (Tilburg University)

  • van Aert, Robbie Cornelis Maria
  • van Assen, Marcel A. L. M.

Abstract

Publication bias remains to be a great challenge when conducting a meta-analysis. It may result in overestimated effect sizes, increased frequency of false positives, and over- or underestimation of the effect size heterogeneity parameter. A new method is introduced, Bayesian Meta-Analytic Snapshot (BMAS), which evaluates both effect size and its heterogeneity and corrects for potential publication bias. It evaluates the probability of the true effect size being zero, small, medium or large, and the probability of true heterogeneity being zero, small, medium or large. This approach, which provides an intuitive evaluation of uncertainty in the evaluation of effect size and heterogeneity, is illustrated with a real-data example, a simulation study, and a Shiny web application of BMAS.

Suggested Citation

  • Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:avkgj
    DOI: 10.31219/osf.io/avkgj
    as

    Download full text from publisher

    File URL: https://osf.io/download/6053162990bb4e006614f8cb/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/avkgj?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sue Duval & Richard Tweedie, 2000. "Trim and Fill: A Simple Funnel-Plot–Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis," Biometrics, The International Biometric Society, vol. 56(2), pages 455-463, June.
    2. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    3. Viechtbauer, Wolfgang, 2010. "Conducting Meta-Analyses in R with the metafor Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i03).
    4. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    5. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    6. Dan Jackson, 2007. "Assessing the Implications of Publication Bias for Two Popular Estimates of between-Study Variance in Meta-Analysis," Biometrics, The International Biometric Society, vol. 63(1), pages 187-193, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robbie C M van Aert & Jelte M Wicherts & Marcel A L M van Assen, 2019. "Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-32, April.
    2. Bart Verkuil & Serpil Atasayi & Marc L Molendijk, 2015. "Workplace Bullying and Mental Health: A Meta-Analysis on Cross-Sectional and Longitudinal Data," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-16, August.
    3. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    4. Georgiou, George K. & Guo, Kan & Naveenkumar, Nithya & Vieira, Ana Paula Alves & Das, J.P., 2020. "PASS theory of intelligence and academic achievement: A meta-analytic review," Intelligence, Elsevier, vol. 79(C).
    5. Nian-Feng Wan & Liwan Fu & Matteo Dainese & Yue-Qing Hu & Lars Pødenphant Kiær & Forest Isbell & Christoph Scherber, 2022. "Plant genetic diversity affects multiple trophic levels and trophic interactions," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    6. Kim, Yeolib & Kim, Seung Hyun & Peterson, Robert A. & Choi, Jeonghye, 2023. "Privacy concern and its consequences: A meta-analysis," Technological Forecasting and Social Change, Elsevier, vol. 196(C).
    7. K. Praveen Parboteeah & Matthias Weiss & Martin Hoegl, 2024. "Ethical Climates Across National Contexts: A Meta-Analytical Investigation," Journal of Business Ethics, Springer, vol. 189(3), pages 573-590, January.
    8. Woodley of Menie, Michael A. & Peñaherrera-Aguirre, Mateo & Sarraf, Matthew A., 2022. "Signs of a Flynn effect in rodents? Secular differentiation of the manifold of general cognitive ability in laboratory mice (Mus musculus) and Norwegian rats (Rattus norvegicus) over a century—Results," Intelligence, Elsevier, vol. 95(C).
    9. Stav Fainshmidt & Amir Pezeshkan & M. Lance Frazier & Anil Nair & Edward Markowski, 2016. "Dynamic Capabilities and Organizational Performance: A Meta-Analytic Evaluation and Extension," Journal of Management Studies, Wiley Blackwell, vol. 53(8), pages 1348-1380, December.
    10. Jaskiewicz, Peter & Block, Joern & Wagner, Dominik & Carney, Michael & Hansen, Christopher, 2021. "How do cross-country differences in institutional trust and trust in family explain the mixed performance effects of family management? A meta-analysis," Journal of World Business, Elsevier, vol. 56(5).
    11. Anouk Decuypere & Wilmar Schaufeli, 2021. "Exploring the Leadership–Engagement Nexus: A Moderated Meta-Analysis and Review of Explaining Mechanisms," IJERPH, MDPI, vol. 18(16), pages 1-34, August.
    12. Haiko Schurz & Michelle Daya & Marlo Möller & Eileen G Hoal & Muneeb Salie, 2015. "TLR1, 2, 4, 6 and 9 Variants Associated with Tuberculosis Susceptibility: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-24, October.
    13. van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2018. "P-uniform," MetaArXiv zqjr9, Center for Open Science.
    14. Kadykalo, Andrew N. & Findlay, C. Scott, 2016. "The flow regulation services of wetlands," Ecosystem Services, Elsevier, vol. 20(C), pages 91-103.
    15. Lang, Jessica & Ochsmann, Elke & Kraus, Thomas & Lang, Jonas W.B., 2012. "Psychosocial work stressors as antecedents of musculoskeletal problems: A systematic review and meta-analysis of stability-adjusted longitudinal studies," Social Science & Medicine, Elsevier, vol. 75(7), pages 1163-1174.
    16. David B. Yaden & Cassondra L. Batz-Barbarich & Vincent Ng & Hoda Vaziri & Jessica N. Gladstone & James O. Pawelski & Louis Tay, 2022. "A Meta-Analysis of Religion/Spirituality and Life Satisfaction," Journal of Happiness Studies, Springer, vol. 23(8), pages 4147-4163, December.
    17. Sergio Nolazco & Kaspar Delhey & Shinichi Nakagawa & Anne Peters, 2022. "Ornaments are equally informative in male and female birds," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    18. Friederike Teetzen & Paul-Christian Bürkner & Sabine Gregersen & Sylvie Vincent-Höper, 2022. "The Mediating Effects of Work Characteristics on the Relationship between Transformational Leadership and Employee Well-Being: A Meta-Analytic Investigation," IJERPH, MDPI, vol. 19(5), pages 1-26, March.
    19. Augusteijn, Hilde & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2017. "The Effect of Publication Bias on the Assessment of Heterogeneity," OSF Preprints gv25c, Center for Open Science.
    20. Tsung-Yuan Hsiao & Wen-Ta Tseng, 2022. "A Meta-Analysis of Test-Retest Reliability in Language Anxiety Research: Is Language Anxiety Stable or Variable?," SAGE Open, , vol. 12(4), pages 21582440221, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:avkgj. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.