Advanced Search
MyIDEAS: Login

The Impact of Effect Size Heterogeneity on Meta-Analysis: A Monte Carlo Experiment

Contents:

Author Info

  • Mark J. Koetse

    ()
    (Vrije Universiteit Amsterdam)

  • Raymond J.G.M. Florax

    ()
    (Purdue University, and Vrije Universiteit Amsterdam)

  • Henri L.F. de Groot

    ()
    (Vrije Universiteit Amsterdam)

Abstract

In this paper we use Monte Carlo simulation to investigate the impact of effect size heterogeneity on the results of a meta-analysis. Specifically, we address the small sample behaviour of the OLS, the fixed effects regression and the mixed effects meta-estimators under three alternative scenarios of effect size heterogeneity. We distinguish heterogeneity in effect size variance, heterogeneity due to a varying true underlying effect across primary studies, and heterogeneity due to a non-systematic impact of omitted variable bias in primary studies. Our results show that the mixed effects estimator is to be preferred to the other two estimators in the first two situations. However, in the presence of random effect size variation due to a non-systematic impact of omitted variable bias, using the mixed effects estimator may be suboptimal. We also address the impact of sample size and show that meta-analysis sample size is far more effective in reducing meta-estimator variance and increasing the power of hypothesis testing than primary study sample size.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://papers.tinbergen.nl/07052.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Tinbergen Institute in its series Tinbergen Institute Discussion Papers with number 07-052/3.

as in new window
Length:
Date of creation: 16 Jul 2007
Date of revision:
Handle: RePEc:dgr:uvatin:20070052

Contact details of provider:
Web page: http://www.tinbergen.nl

Related research

Keywords: Effect size heterogeneity; meta-analysis; Monte Carlo simulation; fixed effects regression estimator; mixed effects estimator;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Mark J. Koetse & Raymond J.G.M. Florax & Henri L.F. de Groot, 2005. "Correcting for Primary Study Misspecifications in Meta-Analysis," Tinbergen Institute Discussion Papers 05-029/3, Tinbergen Institute, revised 31 Jan 2013.
  2. Robert Goldfarb, 1995. "The economist-as-audience needs a methodology of plausible inference," Journal of Economic Methodology, Taylor & Francis Journals, vol. 2(2), pages 201-222.
  3. Julio Sánchez-Meca & Fulgencio Marín-Martínez, 1997. "Homogeneity tests in meta-analysis: a Monte Carlo comparison of statistical power and Type I error," Quality & Quantity: International Journal of Methodology, Springer, vol. 31(4), pages 385-399, November.
  4. Gorg, Holger & Strobl, Eric, 2001. "Multinational Companies and Productivity Spillovers: A Meta-analysis," Economic Journal, Royal Economic Society, vol. 111(475), pages F723-39, November.
  5. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer.
  6. Card, David & Krueger, Alan B, 1995. "Time-Series Minimum-Wage Studies: A Meta-analysis," American Economic Review, American Economic Association, vol. 85(2), pages 238-43, May.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Jon Nelson & Peter Kennedy, 2009. "The Use (and Abuse) of Meta-Analysis in Environmental and Natural Resource Economics: An Assessment," Environmental & Resource Economics, European Association of Environmental and Resource Economists, vol. 42(3), pages 345-377, March.
  2. Melo, Patricia C. & Graham, Daniel J. & Noland, Robert B., 2009. "A meta-analysis of estimates of urban agglomeration economies," Regional Science and Urban Economics, Elsevier, vol. 39(3), pages 332-342, May.
  3. Ceren Ozgen & Peter Nijkamp & Jacques Poot, 2010. "The effect of migration on income growth and convergence: Meta-analytic evidence," Papers in Regional Science, Wiley Blackwell, vol. 89(3), pages 537-561, 08.
  4. Tomáš Havránek, 2009. "Rose Effect and the Euro: The Magic is Gone," Working Papers IES 2009/20, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Aug 2009.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:dgr:uvatin:20070052. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Antoine Maartens (+31 626 - 160 892)).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.