IDEAS home Printed from https://ideas.repec.org/a/zbw/ifweej/201511.html
   My bibliography  Save this article

Meta-analysis in a nutshell: Techniques and general findings

Author

Listed:
  • Paldam, Martin

Abstract

The purpose of this article is to introduce the technique and main findings of meta-analysis to the reader, who is unfamiliar with the field and has the usual objections. A meta-analysis is a quantitative survey of a literature reporting estimates of the same parameter. The funnel showing the distribution of the estimates is normally amazingly wide given their t-ratios. Little of the variation can be explained by the quality of the journal (as measured by its impact factor) or by the estimator used. The funnel has often asymmetries consistent with the most likely priors of the researchers, giving a publication bias.

Suggested Citation

  • Paldam, Martin, 2015. "Meta-analysis in a nutshell: Techniques and general findings," Economics - The Open-Access, Open-Assessment E-Journal, Kiel Institute for the World Economy (IfW), vol. 9, pages 1-14.
  • Handle: RePEc:zbw:ifweej:201511
    DOI: 10.5018/economics-ejournal.ja.2015-11
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.5018/economics-ejournal.ja.2015-11
    Download Restriction: no

    File URL: https://www.econstor.eu/bitstream/10419/109673/1/822413051.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.5018/economics-ejournal.ja.2015-11?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Chris Doucouliagos, 1995. "Worker Participation and Productivity in Labor-Managed and Participatory Capitalist Firms: A Meta-Analysis," ILR Review, Cornell University, ILR School, vol. 49(1), pages 58-77, October.
    2. Tseday Jemaneh Mekasha & Finn Tarp, 2013. "Aid and Growth: What Meta-Analysis Reveals," Journal of Development Studies, Taylor & Francis Journals, vol. 49(4), pages 564-583, April.
    3. Summers, Lawrence H, 1991. " The Scientific Illusion in Empirical Macroeconomics," Scandinavian Journal of Economics, Wiley Blackwell, vol. 93(2), pages 129-148.
    4. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    5. Martin Paldam, 2013. "Regression Costs Fall, Mining Ratios Rise, Publication Bias Looms, and Techniques Get Fancier: Reflections on Some Trends in Empirical Macroeconomics," Econ Journal Watch, Econ Journal Watch, vol. 10(2), pages 136-156, May.
    6. Hristos Doucouliagos & Martin Paldam, 2013. "The Robust Result in Meta-analysis of Aid Effectiveness: A Response to Mekasha and Tarp," Journal of Development Studies, Taylor & Francis Journals, vol. 49(4), pages 584-587, April.
    7. T. D. Stanley, 2008. "Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(1), pages 103-127, February.
    8. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    9. Card, David & Krueger, Alan B, 1995. "Time-Series Minimum-Wage Studies: A Meta-analysis," American Economic Review, American Economic Association, vol. 85(2), pages 238-243, May.
    10. Reed, W. Robert, 2015. "A Monte Carlo analysis of alternative meta-analysis estimators in the presence of publication bias," Economics - The Open-Access, Open-Assessment E-Journal, Kiel Institute for the World Economy (IfW), vol. 9, pages 1-40.
    11. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    12. Martin Paldam & Laurent Callot, 2010. "Natural funnel asymmetries. A simulation analysis of the three basic tools of meta analysis," Economics Working Papers 2010-01, Department of Economics and Business Economics, Aarhus University.
    13. Tseday Jemaneh Mekasha & Finn Tarp, 2013. "Aid and Growth: What Meta-Analysis Reveals," Journal of Development Studies, Taylor & Francis Journals, vol. 49(4), pages 564-583, April.
    14. Stephen B. Jarrell & T. D. Stanley, 1990. "A Meta-Analysis of the Union-Nonunion Wage Gap," ILR Review, Cornell University, ILR School, vol. 44(1), pages 54-67, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Choumert Nkolo, Johanna & Combes Motel, Pascale & Guegang Djimeli, Charlain, 2018. "Income-generating Effects of Biofuel Policies: A Meta-analysis of the CGE Literature," Ecological Economics, Elsevier, vol. 147(C), pages 230-242.
    2. Nelson, Jon Paul, 2020. "Fixed-effect versus random-effects meta-analysis in economics: A study of pass-through rates for alcohol beverage excise taxes," Economics Discussion Papers 2020-1, Kiel Institute for the World Economy (IfW).
    3. Thomas R. Dyckman, 2016. "Significance Testing: We Can Do Better," Abacus, Accounting Foundation, University of Sydney, vol. 52(2), pages 319-342, June.
    4. Paldam, Martin, 2018. "A model of the representative economist, as researcher and policy advisor," European Journal of Political Economy, Elsevier, vol. 54(C), pages 5-15.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
    2. Stephan B. Bruns, 2016. "The Fragility of Meta-Regression Models in Observational Research," MAGKS Papers on Economics 201603, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    3. Stephan B. Bruns, 2013. "Identifying Genuine Effects in Observational Research by Means of Meta-Regressions," Jena Economic Research Papers 2013-040, Friedrich-Schiller-University Jena.
    4. Stephan B. Bruns, 2017. "Meta-Regression Models and Observational Research," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 79(5), pages 637-653, October.
    5. Paldam, Martin, 2018. "A model of the representative economist, as researcher and policy advisor," European Journal of Political Economy, Elsevier, vol. 54(C), pages 5-15.
    6. Nazila Alinaghi & W. Robert Reed, 2021. "Taxes and Economic Growth in OECD Countries: A Meta-analysis," Public Finance Review, , vol. 49(1), pages 3-40, January.
    7. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    8. Brodeur, Abel & Cook, Nikolai & Heyes, Anthony, 2018. "Methods Matter: P-Hacking and Causal Inference in Economics," IZA Discussion Papers 11796, Institute of Labor Economics (IZA).
    9. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    10. Stephan B. Bruns & David I. Stern, 2019. "Lag length selection and p-hacking in Granger causality testing: prevalence and performance of meta-regression models," Empirical Economics, Springer, vol. 56(3), pages 797-830, March.
    11. Rene Tapsoba & Hippolyte W. Balima & Eric G. Kilama, 2017. "Settling the Inflation Targeting Debate: Lights from a Meta-Regression Analysis," IMF Working Papers 2017/213, International Monetary Fund.
    12. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    13. Jianhua Duan & Kuntal K. Das & Laura Meriluoto & W. Robert Reed, 2019. "Spillovers and Exports: A Meta-Analysis," Working Papers in Economics 19/03, University of Canterbury, Department of Economics and Finance.
    14. Jarle Møen & Helge Sandvig Thorsen, 2017. "Publication Bias in the Returns to R&D Literature," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 8(3), pages 987-1013, September.
    15. Martin Paldam, 2013. "Regression Costs Fall, Mining Ratios Rise, Publication Bias Looms, and Techniques Get Fancier: Reflections on Some Trends in Empirical Macroeconomics," Econ Journal Watch, Econ Journal Watch, vol. 10(2), pages 136-156, May.
    16. Tomas Havranek & Anna Sokolova, 2020. "Do Consumers Really Follow a Rule of Thumb? Three Thousand Estimates from 144 Studies Say 'Probably Not'," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 35, pages 97-122, January.
    17. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    18. Amikuzuno, Joseph & Ogundari, Kolawole, 2012. "The Contribution of Agricultural Economics to Price transmission Analysis and Market Policy in Sub-Sahara Africa: What Does the Literature Say?," 86th Annual Conference, April 16-18, 2012, Warwick University, Coventry, UK 134754, Agricultural Economics Society.
    19. Fabo, Brian & Jancokova, Martina & Kempf, Elisabeth & Pástor, Lubos, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," CEPR Discussion Papers 15449, C.E.P.R. Discussion Papers.
    20. Bruns, Stephan B. & Ioannidis, John P.A., 2020. "Determinants of economic growth: Different time different answer?," Journal of Macroeconomics, Elsevier, vol. 63(C).

    More about this item

    Keywords

    meta-analysis; selection of regressions; publication bias;
    All these keywords.

    JEL classification:

    • B4 - Schools of Economic Thought and Methodology - - Economic Methodology
    • C2 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Meta-Analysis in Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:ifweej:201511. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/iwkiede.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/iwkiede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.