IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/118433.html
   My bibliography  Save this paper

Do better journals publish better estimates?

Author

Listed:
  • Slichter, David
  • Tran, Nhan

Abstract

Are estimates typically closer to the true parameter value when those estimates are published in highly-ranked economics journals? Using 14,387 published estimates from 24 large literatures, we find that, within literatures, the mean and variance of parameter estimates have little or no correlation with journal rank. Therefore, regardless of what the true parameter value is that a literature is attempting to estimate, it cannot be that estimates in higher-ranked journals are on average noticeably closer to it. We discuss possible explanations and implications.

Suggested Citation

  • Slichter, David & Tran, Nhan, 2023. "Do better journals publish better estimates?," MPRA Paper 118433, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:118433
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/118433/1/Better_journals.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter Nijkamp & Jacques Poot, 2005. "The Last Word on the Wage Curve?," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 421-450, July.
    2. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    3. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    4. Havranek, Tomas & Irsova, Zuzana & Janda, Karel & Zilberman, David, 2015. "Selective reporting and the social cost of carbon," Energy Economics, Elsevier, vol. 51(C), pages 394-406.
    5. Simonetta LONGHI & Peter NIJKAMP & Jacques POOT, 2008. "Meta-Analysis Of Empirical Evidence On The Labour Market Impacts Of Immigration," Region et Developpement, Region et Developpement, LEAD, Universite du Sud - Toulon Var, vol. 27, pages 161-191.
    6. Xinxin Ma & Ichiro Iwasaki, 2021. "Return to schooling in China: a large meta-analysis," Education Economics, Taylor & Francis Journals, vol. 29(4), pages 379-410, July.
    7. Jon Nelson, 2013. "Meta-analysis of alcohol price and income elasticities – with corrections for publication bias," Health Economics Review, Springer, vol. 3(1), pages 1-10, December.
    8. Tomas Havranek, 2013. "Publication Bias in Measuring Intertemporal Substitution," Working Papers IES 2013/15, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Oct 2013.
    9. Carina Neisser, 2021. "The Elasticity of Taxable Income: A Meta-Regression Analysis [The top 1% in international and historical perspective]," The Economic Journal, Royal Economic Society, vol. 131(640), pages 3365-3391.
    10. Chris Doucouliagos, 2005. "Publication Bias in the Economic Freedom and Economic Growth Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 367-387, July.
    11. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    12. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    13. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    14. Tomáš Havránek, 2015. "Measuring Intertemporal Substitution: The Importance Of Method Choices And Selective Reporting," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1180-1204, December.
    15. Nicolas Fleury & Fabrice Gilles, 2018. "The intergenerational transmission of education. A meta-regression analysis," Education Economics, Taylor & Francis Journals, vol. 26(6), pages 557-573, November.
    16. Hristos Doucouliagos & T. D. Stanley, 2009. "Publication Selection Bias in Minimum‐Wage Research? A Meta‐Regression Analysis," British Journal of Industrial Relations, London School of Economics, vol. 47(2), pages 406-428, June.
    17. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    18. Tomas Havranek & Zuzana Irsova & Olesia Zeynalova, 2018. "Tuition Fees and University Enrolment: A Meta‐Regression Analysis," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 80(6), pages 1145-1184, December.
    19. Tomas Havranek & Anna Sokolova, 2020. "Do Consumers Really Follow a Rule of Thumb? Three Thousand Estimates from 144 Studies Say 'Probably Not'," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 35, pages 97-122, January.
    20. Young, Alwyn, 2022. "Consistency without Inference: Instrumental Variables in Practical Application," European Economic Review, Elsevier, vol. 147(C).
    21. Havranek, Tomas & Irsova, Zuzana & Janda, Karel, 2012. "Demand for gasoline is more price-inelastic than commonly thought," Energy Economics, Elsevier, vol. 34(1), pages 201-207.
    22. Tomas Havranek & Zuzana Irsova & Tomas Vlach, 2018. "Measuring the Income Elasticity of Water Demand: The Importance of Publication and Endogeneity Biases," Land Economics, University of Wisconsin Press, vol. 94(2), pages 259-283.
    23. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    24. Ma, Xinxin & Iwasaki, Ichiro, 2021. "A Systematic Review of Return to Schooling in China," Economic Review, Hitotsubashi University, vol. 72(1), pages 38-58, January.
    25. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Publisher Correction: Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1215-1215, November.
    26. Binyam Afewerk Demena, 2015. "Publication bias in FDI spillovers in developing countries: a meta-regression analysis," Applied Economics Letters, Taylor & Francis Journals, vol. 22(14), pages 1170-1174, September.
    27. Lichter, Andreas & Peichl, Andreas & Siegloch, Sebastian, 2015. "The own-wage elasticity of labor demand: A meta-regression analysis," European Economic Review, Elsevier, vol. 80(C), pages 94-119.
    28. Bajzik, Josef & Havranek, Tomas & Irsova, Zuzana & Schwarz, Jiri, 2020. "Estimating the Armington elasticity: The importance of study design and publication bias," Journal of International Economics, Elsevier, vol. 127(C).
    29. Gallet, Craig A. & Doucouliagos, Hristos, 2017. "The impact of healthcare spending on health outcomes: A meta-regression analysis," Social Science & Medicine, Elsevier, vol. 179(C), pages 9-17.
    30. Xindong Xue & Mingmei Cheng & Wangyongxin Zhang, 2021. "Does Education Really Improve Health? A Meta‐Analysis," Journal of Economic Surveys, Wiley Blackwell, vol. 35(1), pages 71-105, February.
    31. Young, Alwyn, 2022. "Consistency without inference: instrumental variables in practical application," LSE Research Online Documents on Economics 115011, London School of Economics and Political Science, LSE Library.
    32. Dahl, Carol A., 2012. "Measuring global gasoline and diesel price and income elasticities," Energy Policy, Elsevier, vol. 41(C), pages 2-13.
    33. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    2. Cazachevici, Alina & Havranek, Tomas & Horvath, Roman, 2020. "Remittances and economic growth: A meta-analysis," World Development, Elsevier, vol. 134(C).
    3. Zigraiova, Diana & Havranek, Tomas & Irsova, Zuzana & Novak, Jiri, 2021. "How puzzling is the forward premium puzzle? A meta-analysis," European Economic Review, Elsevier, vol. 134(C).
    4. Havranek, Tomas & Horvath, Roman & Zeynalov, Ayaz, 2016. "Natural Resources and Economic Growth: A Meta-Analysis," World Development, Elsevier, vol. 88(C), pages 134-151.
    5. Ali Elminejad & Tomas Havranek & Roman Horvath & Zuzana Irsova, 2023. "Intertemporal Substitution in Labor Supply: A Meta-Analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 51, pages 1095-1113, December.
    6. Gechert, Sebastian & Heimberger, Philipp, 2022. "Do corporate tax cuts boost economic growth?," European Economic Review, Elsevier, vol. 147(C).
    7. repec:wly:econjl:v::y:2017:i:605:p:f236-f265 is not listed on IDEAS
    8. Jindrich Matousek & Tomas Havranek & Zuzana Irsova, 2022. "Individual discount rates: a meta-analysis of experimental evidence," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 318-358, February.
    9. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    10. Sebastian Gechert & Tomas Havranek & Zuzana Irsova & Dominika Kolcunova, 2022. "Measuring Capital-Labor Substitution: The Importance of Method Choices and Publication Bias," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 45, pages 55-82, July.
    11. Amar Anwar & Ichiro Iwasaki, 2022. "Institutions and FDI from BRICS countries: a meta-analytic review," Empirical Economics, Springer, vol. 63(1), pages 417-468, July.
    12. Mojmir Hampl & Tomas Havranek, 2020. "Central Bank Equity as an Instrument of Monetary Policy," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 62(1), pages 49-68, March.
    13. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    14. Bensch, Gunther & Ankel-Peters, Jörg & Vance, Colin, 2023. "Spotlight on Researcher Decisions – Infrastructure Evaluation, Instrumental Variables, and Specification Screening," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277703, Verein für Socialpolitik / German Economic Association.
    15. Michal Hlavacek & Ilgar Ismayilov, 2022. "Meta-analysis: Fiscal Multiplier," Working Papers IES 2022/07, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2022.
    16. Irsova, Zuzana & Bom, Pedro Ricardo Duarte & Havranek, Tomas & Rachinger, Heiko, 2023. "Spurious Precision in Meta-Analysis," MetaArXiv 3qp2w, Center for Open Science.
    17. Ankel-Peters, Jörg & Vance, Colin & Bensch, Gunther, 2022. "Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening," OSF Preprints sw6kd, Center for Open Science.
    18. Bajzik, Josef, 2021. "Trading volume and stock returns: A meta-analysis," International Review of Financial Analysis, Elsevier, vol. 78(C).
    19. Demena, B.A., 2021. "Effectiveness of export promotion programmes," ISS Working Papers - General Series 688, International Institute of Social Studies of Erasmus University Rotterdam (ISS), The Hague.
    20. Kudela, Peter & Havranek, Tomas & Herman, Dominik & Irsova, Zuzana, 2020. "Does daylight saving time save electricity? Evidence from Slovakia," Energy Policy, Elsevier, vol. 137(C).
    21. Gric, Zuzana & Bajzík, Josef & Badura, Ondřej, 2023. "Does sentiment affect stock returns? A meta-analysis across survey-based measures," International Review of Financial Analysis, Elsevier, vol. 89(C).

    More about this item

    Keywords

    Meta-analysis; scientific methods; publication; science of science;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:118433. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.