IDEAS home Printed from https://ideas.repec.org/p/fau/wpaper/wp2023_05.html

Spurious Precision in Meta-Analysis

Author

Listed:
  • Zuzana Irsova

    (Charles University, Prague)

  • Pedro R. D. Bom

    (University of Deusto, Bilbao)

  • Tomas Havranek

    (Charles University, Prague & Centre for Economic Policy Research, London & Meta-Research Innovation Center, Stanford)

  • Heiko Rachinger

    (University of the Balearic Islands, Palma)

Abstract

Meta-analysis upweights studies reporting lower standard errors and hence more preci- sion. But in empirical practice, notably in observational research, precision is not given to the researcher. Precision must be estimated, and thus can be p-hacked to achieve statistical significance. Simulations show that a modest dose of spurious precision creates a formidable problem for inverse-variance weighting and bias-correction methods based on the funnel plot. Selection models fail to solve the problem, and the simple mean can beat sophisticated estimators. Cures to publication bias may become worse than the disease. We introduce an approach that surmounts spuriousness: the Meta-Analysis Instrumental Variable Estimator (MAIVE), which employs inverse sample size as an instrument for reported variance.

Suggested Citation

  • Zuzana Irsova & Pedro R. D. Bom & Tomas Havranek & Heiko Rachinger, 2023. "Spurious Precision in Meta-Analysis," Working Papers IES 2023/05, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Feb 2023.
    • Handle: RePEc:fau:wpaper:wp2023_05
    as

    Download full text from publisher

    File URL: https://ies.fsv.cuni.cz/en/spurious-precision-meta-analysis
    Download Restriction: no
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Intertemporal Substitution in Labor Supply: A Meta-Analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 1095-1113, December.
    2. Zuzana Irsova & Hristos Doucouliagos & Tomas Havranek & T. D. Stanley, 2024. "Meta‐analysis of social science research: A practitioner's guide," Journal of Economic Surveys, Wiley Blackwell, vol. 38(5), pages 1547-1566, December.
    3. Schneider, Florian, 2024. "Do robots boost productivity? A quantitative meta-study," MPRA Paper 123392, University Library of Munich, Germany.
    4. Pazzona, Matteo, 2024. "Revisiting the Income Inequality-Crime Puzzle," World Development, Elsevier, vol. 176(C).
    5. Matej Opatrny & Tomas Havranek & Zuzana Irsova & Milan Scasny, 2023. "Publication Bias and Model Uncertainty in Measuring the Effect of Class Size on Achievement," Working Papers IES 2023/19, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2023.
    6. Rodolfo Campos & Mario Larch & Jacopo Timini & Elena Vidal & Yoto Yotov, 2024. "Does the WTO Promote Trade? A Meta-analysis," School of Economics Working Paper Series 2024-11, LeBow College of Business, Drexel University.
    7. Tersoo David Iorngurum, 2023. "Method Versus Cross-Country Heterogeneity in the Exchange Rate Pass-Through," Working Papers IES 2023/16, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2023.
    8. Ichiro Iwasaki & Evžen Kočenda, 2024. "Quest for the general effect size of finance on growth: a large meta-analysis of worldwide studies," Empirical Economics, Springer, vol. 66(6), pages 2659-2722, June.
    9. T. D. Stanley & Hristos Doucouliagos & Tomas Havranek, 2023. "Reducing the Biases of the Conventional Meta-Analysis of Correlations," Working Papers IES 2023/34, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Dec 2023.
    10. Marvin Schütt, 2024. "Wind Turbines and Property Values: A Meta-Regression Analysis," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 87(1), pages 1-43, January.
    11. Kroupova, Katerina & Havranek, Tomas & Irsova, Zuzana, 2024. "Student Employment and Education: A Meta-Analysis," Economics of Education Review, Elsevier, vol. 100(C).
    12. Yanxia Yu & Chenfei Qu & Tom Coupé & Mathilda Featherston-Lardeux & Andreas Loeschel & Arne R. Weiss & Da Zhang, 2025. "Did China’s Pilot Emissions Trading Scheme Reduce CO2 Emissions? Evidence from a Meta-Analysis," Working Papers in Economics 25/04, University of Canterbury, Department of Economics and Finance.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fau:wpaper:wp2023_05. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Natalie Svarcova (email available below). General contact details of provider: https://edirc.repec.org/data/icunicz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.