This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

The Impact of Effect Size Heterogeneity on Meta-Analysis: A Monte Carlo Experiment

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Mark J. Koetse () (Vrije Universiteit Amsterdam)
Raymond J.G.M. Florax () (Purdue University, and Vrije Universiteit Amsterdam)
Henri L.F. de Groot () (Vrije Universiteit Amsterdam)

Additional information is available for the following registered author(s):

Abstract

In this paper we use Monte Carlo simulation to investigate the impact of effect size heterogeneity on the results of a meta-analysis. Specifically, we address the small sample behaviour of the OLS, the fixed effects regression and the mixed effects meta-estimators under three alternative scenarios of effect size heterogeneity. We distinguish heterogeneity in effect size variance, heterogeneity due to a varying true underlying effect across primary studies, and heterogeneity due to a non-systematic impact of omitted variable bias in primary studies. Our results show that the mixed effects estimator is to be preferred to the other two estimators in the first two situations. However, in the presence of random effect size variation due to a non-systematic impact of omitted variable bias, using the mixed effects estimator may be suboptimal. We also address the impact of sample size and show that meta-analysis sample size is far more effective in reducing meta-estimator variance and increasing the power of hypothesis testing than primary study sample size.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.tinbergen.nl/discussionpapers/07052.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Tinbergen Institute in its series Tinbergen Institute Discussion Papers with number 07-052/3.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: 16 Jul 2007
Date of revision:
Handle: RePEc:dgr:uvatin:20070052

Contact details of provider:
Web page: http://www.tinbergen.nl/

For technical questions regarding this item, or to correct its listing, contact: (Walther Schoonenberg).

Related research
Keywords: Effect size heterogeneity; meta-analysis; Monte Carlo simulation; fixed effects regression estimator; mixed effects estimator;

Find related papers by JEL classification:
C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Hypothesis Testing
C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Statistical Simulation Methods
C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General

This paper has been announced in the following NEP Reports:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. T. D. Stanley, 2001. "Wheat from Chaff: Meta-analysis as Quantitative Literature Review," Journal of Economic Perspectives, American Economic Association, vol. 15(3), pages 131-150, Summer. [Downloadable!] (restricted)
  2. Gorg, Holger & Strobl, Eric, 2001. "Multinational Companies and Productivity Spillovers: A Meta-analysis," Economic Journal, Royal Economic Society, vol. 111(475), pages F723-39, November. [Downloadable!] (restricted)
  3. Mark J. Koetse & Raymond J.G.M. Florax & Henri L.F. de Groot, 2005. "Correcting for Primary Study Misspecifications in Meta-Analysis," Tinbergen Institute Discussion Papers 05-029/3, Tinbergen Institute, revised 17 Jun 2009. [Downloadable!]
  4. Card, David & Krueger, Alan B, 1995. "Time-Series Minimum-Wage Studies: A Meta-analysis," American Economic Review, American Economic Association, vol. 85(2), pages 238-43, May. [Downloadable!] (restricted)
  5. Goldfarb, Robert S, 1995. "The Economist-as-Audience Needs a Methodology of Plausible Inference," Journal of Economic Methodology, Taylor and Francis Journals, vol. 2(2), pages 201-22, December.
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Jon Nelson & Peter Kennedy, 2009. "The Use (and Abuse) of Meta-Analysis in Environmental and Natural Resource Economics: An Assessment," Environmental & Resource Economics, European Association of Environmental and Resource Economists, vol. 42(3), pages 345-377, March. [Downloadable!] (restricted)
Statistics
Access and download statistics

Did you know? Want to help out with this project? Look for volunteer opportunities.

This page was last updated on 2009-11-26.


This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.