IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/118433.html

Do better journals publish better estimates?

Author

Listed:
  • Slichter, David
  • Tran, Nhan

Abstract

Are estimates typically closer to the true parameter value when those estimates are published in highly-ranked economics journals? Using 14,387 published estimates from 24 large literatures, we find that, within literatures, the mean and variance of parameter estimates have little or no correlation with journal rank. Therefore, regardless of what the true parameter value is that a literature is attempting to estimate, it cannot be that estimates in higher-ranked journals are on average noticeably closer to it. We discuss possible explanations and implications.

Suggested Citation

  • Slichter, David & Tran, Nhan, 2023. "Do better journals publish better estimates?," MPRA Paper 118433, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:118433
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/118433/1/Better_journals.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Binyam Afewerk Demena, 2015. "Publication bias in FDI spillovers in developing countries: a meta-regression analysis," Applied Economics Letters, Taylor & Francis Journals, vol. 22(14), pages 1170-1174, September.
    2. Xinxin Ma & Ichiro Iwasaki, 2021. "Return to schooling in China: a large meta-analysis," Education Economics, Taylor & Francis Journals, vol. 29(4), pages 379-410, July.
    3. Lichter, Andreas & Peichl, Andreas & Siegloch, Sebastian, 2015. "The own-wage elasticity of labor demand: A meta-regression analysis," European Economic Review, Elsevier, vol. 80(C), pages 94-119.
    4. Nick Huntington-Klein & Claus Pörtner & Yubraj Acharya & Matus Adamkovic & Joop Adema & Lameck Ondieki Agasa & Imtiaz Ahmad & Mevlude Akbulut-Yuksel & Martin Eckhoff Andresen & David Angenendt & José-, 2025. "The Sources of Researcher Variation in Economics," Working Papers hal-05187084, HAL.
    5. Peter Nijkamp & Jacques Poot, 2005. "The Last Word on the Wage Curve?," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 421-450, July.
    6. Bajzik, Josef & Havranek, Tomas & Irsova, Zuzana & Schwarz, Jiri, 2020. "Estimating the Armington elasticity: The importance of study design and publication bias," Journal of International Economics, Elsevier, vol. 127(C).
    7. Simonetta LONGHI & Peter NIJKAMP & Jacques POOT, 2008. "Meta-Analysis Of Empirical Evidence On The Labour Market Impacts Of Immigration," Region et Developpement, Region et Developpement, LEAD, Universite du Sud - Toulon Var, vol. 27, pages 161-191.
    8. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    9. Havranek, Tomas & Irsova, Zuzana & Janda, Karel & Zilberman, David, 2015. "Selective reporting and the social cost of carbon," Energy Economics, Elsevier, vol. 51(C), pages 394-406.
    10. Gallet, Craig A. & Doucouliagos, Hristos, 2017. "The impact of healthcare spending on health outcomes: A meta-regression analysis," Social Science & Medicine, Elsevier, vol. 179(C), pages 9-17.
    11. Jon Nelson, 2013. "Meta-analysis of alcohol price and income elasticities – with corrections for publication bias," Health Economics Review, Springer, vol. 3(1), pages 1-10, December.
    12. Tomas Havranek, 2013. "Publication Bias in Measuring Intertemporal Substitution," Working Papers IES 2013/15, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Oct 2013.
    13. Xindong Xue & Mingmei Cheng & Wangyongxin Zhang, 2021. "Does Education Really Improve Health? A Meta‐Analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 35(1), pages 71-105, February.
    14. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    15. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    16. Chris Doucouliagos, 2005. "Publication Bias in the Economic Freedom and Economic Growth Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 367-387, July.
    17. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    18. Kroupova, Katerina & Havranek, Tomas & Irsova, Zuzana, 2024. "Student Employment and Education: A Meta-Analysis," Economics of Education Review, Elsevier, vol. 100(C).
    19. Nicolas Fleury & Fabrice Gilles, 2018. "The intergenerational transmission of education. A meta-regression analysis," Education Economics, Taylor & Francis Journals, vol. 26(6), pages 557-573, November.
    20. Young, Alwyn, 2022. "Consistency without inference: instrumental variables in practical application," LSE Research Online Documents on Economics 115011, London School of Economics and Political Science, LSE Library.
    21. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    22. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    23. Dahl, Carol A., 2012. "Measuring global gasoline and diesel price and income elasticities," Energy Policy, Elsevier, vol. 41(C), pages 2-13.
    24. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    25. Tomáš Havránek, 2015. "Measuring Intertemporal Substitution: The Importance Of Method Choices And Selective Reporting," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1180-1204, December.
    26. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    27. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    28. Tomas Havranek & Zuzana Irsova & Olesia Zeynalova, 2018. "Tuition Fees and University Enrolment: A Meta‐Regression Analysis," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 80(6), pages 1145-1184, December.
    29. Tomas Havranek & Anna Sokolova, 2020. "Do Consumers Really Follow a Rule of Thumb? Three Thousand Estimates from 144 Studies Say 'Probably Not'," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 35, pages 97-122, January.
    30. Young, Alwyn, 2022. "Consistency without Inference: Instrumental Variables in Practical Application," European Economic Review, Elsevier, vol. 147(C).
    31. Carina Neisser, 2021. "The Elasticity of Taxable Income: A Meta-Regression Analysis [The top 1% in international and historical perspective]," The Economic Journal, Royal Economic Society, vol. 131(640), pages 3365-3391.
    32. Tomas Havranek & Zuzana Irsova & Tomas Vlach, 2018. "Measuring the Income Elasticity of Water Demand: The Importance of Publication and Endogeneity Biases," Land Economics, University of Wisconsin Press, vol. 94(2), pages 259-283.
    33. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    34. Tomas Havranek & Zuzana Irsova & Lubica Laslopova & Olesia Zeynalova, 2024. "Publication and Attenuation Biases in Measuring Skill Substitution," The Review of Economics and Statistics, MIT Press, vol. 106(5), pages 1187-1200, September.
    35. Havranek, Tomas & Irsova, Zuzana & Janda, Karel, 2012. "Demand for gasoline is more price-inelastic than commonly thought," Energy Economics, Elsevier, vol. 34(1), pages 201-207.
    36. Ma, Xinxin & Iwasaki, Ichiro, 2021. "A Systematic Review of Return to Schooling in China," Economic Review, Hitotsubashi University, vol. 72(1), pages 38-58, January.
    37. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Publisher Correction: Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1215-1215, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zuzana Irsova & Hristos Doucouliagos & Tomas Havranek & T. D. Stanley, 2024. "Meta‐analysis of social science research: A practitioner's guide," Journal of Economic Surveys, Wiley Blackwell, vol. 38(5), pages 1547-1566, December.
    2. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2020. "Remittances and economic growth: A meta-analysis," World Development, Elsevier, vol. 134(C).
    3. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    4. Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Intertemporal Substitution in Labor Supply: A Meta-Analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 1095-1113, December.
    5. Zigraiova, Diana & Havranek, Tomas & Irsova, Zuzana & Novak, Jiri, 2021. "How puzzling is the forward premium puzzle? A meta-analysis," European Economic Review, Elsevier, vol. 134(C).
    6. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    7. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    8. repec:osf:osfxxx:sw6kd_v1 is not listed on IDEAS
    9. Anna Sokolova, 2023. "Marginal Propensity to Consume and Unemployment: a Meta-analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 813-846, December.
    10. repec:wly:econjl:v::y:2017:i:605:p:f236-f265 is not listed on IDEAS
    11. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    12. Bensch, Gunther & Ankel-Peters, Jörg & Vance, Colin, 2023. "Spotlight on Researcher Decisions – Infrastructure Evaluation, Instrumental Variables, and Specification Screening," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277703, Verein für Socialpolitik / German Economic Association.
    13. Anna Dreber & Magnus Johannesson, 2025. "A framework for evaluating reproducibility and replicability in economics," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 338-356, April.
    14. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    15. Sebastian Gechert & Bianka Mey & Matej Opatrny & Tomas Havranek & T. D. Stanley & Pedro R. D. Bom & Hristos Doucouliagos & Philipp Heimberger & Zuzana Irsova & Heiko J. Rachinger, 2025. "Conventional wisdom, meta‐analysis, and research revision in economics," Journal of Economic Surveys, Wiley Blackwell, vol. 39(3), pages 980-999, July.
    16. Ankel-Peters, Jörg & Vance, Colin & Bensch, Gunther, 2022. "Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening," OSF Preprints sw6kd, Center for Open Science.
    17. Gric, Zuzana & Bajzík, Josef & Badura, Ondřej, 2023. "Does sentiment affect stock returns? A meta-analysis across survey-based measures," International Review of Financial Analysis, Elsevier, vol. 89(C).
    18. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    19. Havranek, Tomas & Horvath, Roman & Zeynalov, Ayaz, 2016. "Natural Resources and Economic Growth: A Meta-Analysis," World Development, Elsevier, vol. 88(C), pages 134-151.
    20. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    21. Mojmir Hampl & Tomas Havranek, 2020. "Central Bank Equity as an Instrument of Monetary Policy," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 62(1), pages 49-68, March.
    22. Anthony Doucouliagos & Hristos Doucouliagos & T. D. Stanley, 2024. "Power and bias in industrial relations research," British Journal of Industrial Relations, London School of Economics, vol. 62(1), pages 3-27, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:118433. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.