IDEAS home Printed from https://ideas.repec.org/p/cbt/econwp/25-16.html

Meta-Analyses in Management and Marketing: An Assessment

Author

Listed:

Abstract

Meta-analysis plays a central role in evidence synthesis in management and marketing, yet little evidence exists on how closely published studies adhere to established methodological guidance. This paper provides a structured assessment of 100 meta-analyses published between 2023 and 2025 in top-tier journals. We document key features of current practice, including data scale and structure, estimator and effect-size choices, approaches to detecting and adjusting for publication bias, heterogeneity reporting and exploration, open science practices, and software usage. We find a substantial implementation gap between recommended methods and routine practice. Although most meta-analyses extract multiple effect sizes per primary study, fewer than 40% of those acknowledging dependence employ multilevel or multivariate models. Correlation-based effect sizes dominate but rarely incorporate recommended transformations or weighting strategies designed to avoid known algebraic distortions. Heterogeneity is extreme (median I² > 95%), yet are often only partially reported or explored. While publication bias is commonly tested, fewer than half of studies report bias-adjusted estimates, and low-powered diagnostic tools are frequently relied upon. We conclude by identifying inferential consequences that are particularly salient for the credibility and interpretability of meta-analytic evidence in management and marketing.

Suggested Citation

  • Weilun Wu & W. Robert Reed, 2025. "Meta-Analyses in Management and Marketing: An Assessment," Working Papers in Economics 25/16, University of Canterbury, Department of Economics and Finance.
  • Handle: RePEc:cbt:econwp:25/16
    as

    Download full text from publisher

    File URL: https://repec.canterbury.ac.nz/cbt/econwp/2516.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zuzana Irsova & Pedro R. D. Bom & Tomas Havranek & Heiko Rachinger, 2025. "Spurious precision in meta-analysis of observational research," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    2. repec:osf:metaar:3qp2w_v2 is not listed on IDEAS
    3. James E. Pustejovsky & Elizabeth Tipton, 2018. "Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 672-683, October.
    4. Nosek, BA & Alter, G & Banks, GC & Borsboom, D & Bowman, SD & Breckler, SJ & Buck, S & Chambers, CD & Chin, G & Christensen, G & Contestabile, M & Dafoe, A & Eich, E & Freese, J & Glennerster, R & Gor, 2015. "Promoting an open research culture," Department of Economics, Working Paper Series qt7wh1000s, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    5. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:osf:metaar:3qp2w_v1 is not listed on IDEAS
    2. Erik Johannesson & James A. Ohlson & Sophia Weihuan Zhai, 2024. "The explanatory power of explanatory variables," Review of Accounting Studies, Springer, vol. 29(4), pages 3053-3083, December.
    3. MacKinnon, James G. & Orregaard Nielsen, Morten & Webb, Matthew D., 2017. "Bootstrap and Asymptotic Inference with Multiway Clustering," Queen's Economics Department Working Papers 274712, Queen's University - Department of Economics.
    4. Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
    5. Thomas S. Dee & Emily K. Penner, 2021. "My Brother's Keeper? The Impact of Targeted Educational Supports," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(4), pages 1171-1196, September.
    6. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    7. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    8. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, IZA Network @ LISER.
    9. Alan Manning, 2021. "The Elusive Employment Effect of the Minimum Wage," Journal of Economic Perspectives, American Economic Association, vol. 35(1), pages 3-26, Winter.
    10. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    11. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," IZA Discussion Papers 15478, IZA Network @ LISER.
    12. Adam C. Sales & Ben B. Hansen, 2020. "Limitless Regression Discontinuity," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 143-174, April.
    13. Nazila Alinaghi & W. Robert Reed, 2016. "Meta-Analysis and Publication Bias: How Well Does the FAT-PET-PEESE Procedure Work?," Working Papers in Economics 16/26, University of Canterbury, Department of Economics and Finance.
    14. Mattia Filomena & Matteo Picchio, 2023. "Retirement and health outcomes in a meta‐analytical framework," Journal of Economic Surveys, Wiley Blackwell, vol. 37(4), pages 1120-1155, September.
    15. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    16. Caicedo, Juan D. & Guirado, Carlos & González, Marta C. & Walker, Joan L., 2025. "Sharing, collaborating, and benchmarking to advance travel demand research: A demonstration of short-term ridership prediction," Transport Policy, Elsevier, vol. 171(C), pages 531-541.
    17. Castaing, Pauline & Gazeaud, Jules, 2025. "Do index insurance programs live up to their promises? Aggregating evidence from multiple experiments," Journal of Development Economics, Elsevier, vol. 175(C).
    18. Patrick Vu, 2022. "Can the Replication Rate Tell Us About Publication Bias?," Papers 2206.15023, arXiv.org, revised Jul 2022.
    19. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    20. Roman Horvath & Ali Elminejad & Tomas Havranek, 2020. "Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply," Working Papers IES 2020/32, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Sep 2020.
    21. Bobasu, Alina & Ciccarelli, Matteo & Notarpietro, Alessandro & Ambrocio, Gene & Auer, Simone & Bonfim, Diana & Bottero, Margherita & Brázdik, František & Buss, Ginters & Byrne, David & Casalis, André , 2025. "Monetary policy transmission: a reference guide through ESCB models and empirical benchmarks," Occasional Paper Series 377, European Central Bank.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C19 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Other
    • M00 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - General - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cbt:econwp:25/16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Albert Yee (email available below). General contact details of provider: https://edirc.repec.org/data/decannz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.