Meta-analyzing non-preregistered and preregistered studies

Preregistration is gaining ground in psychology, and a consequence of this is that preregistered studies are more often included in meta-analyses. Preregistered studies mitigate the effect of publication bias in a meta-analysis, because preregistered studies can be located in the registries they were registered in even if they do not get published. However, current meta-analysis methods do not take into account that preregistered studies are less susceptible to publication bias. Traditional methods treat all studies as equivalent while meta-analytic conclusions can be improved by taking advantage of preregistered studies. The goal of this paper is to introduce a new method, the Hybrid Extended Meta-Analysis (HYEMA) method, that takes into account whether a study is preregistered or not to correct for publication bias in only the non-preregistered studies. The proposed method is applied to two meta-analyses on prominent effects in the psychological literature: the red-romance hypothesis and money priming. Applying HYEMA to these meta-analyses shows that the average effect size is substantially closer to zero than of the random-effects meta-analysis model. Two simulation studies tailored to the two applications are also presented to illustrate the method's superior performance compared to the random-effects meta-analysis model when publication bias is present. Hence, I recommend to always apply HYEMA as a sensitivity analysis if a mix of both preregistered and non-preregistered studies are present in a meta-analysis. Software is also developed and described in the paper to facilitate application of the method.