IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v140y2017i3d10.1007_s10584-016-1880-1.html
   My bibliography  Save this article

No evidence of publication bias in climate change science

Author

Listed:
  • Christian Harlos

    (Lund University)

  • Tim C. Edgell

    (Stantec)

  • Johan Hollander

    (Lund University)

Abstract

Non-significant results are less likely to be reported by authors and, when submitted for peer review, are less likely to be published by journal editors. This phenomenon, known collectively as publication bias, is seen in a variety of scientific disciplines and can erode public trust in the scientific method and the validity of scientific theories. Public trust in science is especially important for fields like climate change science, where scientific consensus can influence state policies on a global scale, including strategies for industrial and agricultural management and development. Here, we used meta-analysis to test for biases in the statistical results of climate change articles, including 1154 experimental results from a sample of 120 articles. Funnel plots revealed no evidence of publication bias given no pattern of non-significant results being under-reported, even at low sample sizes. However, we discovered three other types of systematic bias relating to writing style, the relative prestige of journals, and the apparent rise in popularity of this field: First, the magnitude of statistical effects was significantly larger in the abstract than the main body of articles. Second, the difference in effect sizes in abstracts versus main body of articles was especially pronounced in journals with high impact factors. Finally, the number of published articles about climate change and the magnitude of effect sizes therein both increased within 2 years of the seminal report by the Intergovernmental Panel on Climate Change 2007.

Suggested Citation

  • Christian Harlos & Tim C. Edgell & Johan Hollander, 2017. "No evidence of publication bias in climate change science," Climatic Change, Springer, vol. 140(3), pages 375-385, February.
    • Handle: RePEc:spr:climat:v:140:y:2017:i:3:d:10.1007_s10584-016-1880-1
    DOI: 10.1007/s10584-016-1880-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-016-1880-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10584-016-1880-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Adam Corner & Lorraine Whitmarsh & Dimitrios Xenias, 2012. "Uncertainty, scepticism and attitudes towards climate change: biased assimilation and attitude polarisation," Climatic Change, Springer, vol. 114(3), pages 463-478, October.
    2. David Holland, 2007. "Bias and Concealment in the IPCC Process: The “Hockey-Stick†Affair and its Implications," Energy & Environment, , vol. 18(7), pages 951-983, December.
    3. Michal Kicinski, 2013. "Publication Bias in Recent Meta-Analyses," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-1, November.
    4. Daniele Fanelli, 2013. "Positive results receive more citations, but only in some disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 701-709, February.
    5. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Augusteijn, Hilde & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2017. "The Effect of Publication Bias on the Assessment of Heterogeneity," OSF Preprints gv25c, Center for Open Science.
    2. Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
    3. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    4. Douglas Guilbeault & Damon Centola, 2020. "Networked collective intelligence improves dissemination of scientific information regarding smoking risks," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-14, February.
    5. Christian Heise & Joshua M. Pearce, 2020. "From Open Access to Open Science: The Path From Scientific Reality to Open Scientific Communication," SAGE Open, , vol. 10(2), pages 21582440209, May.
    6. András Szeberényi & Tomasz Rokicki & Árpád Papp-Váry, 2022. "Examining the Relationship between Renewable Energy and Environmental Awareness," Energies, MDPI, vol. 15(19), pages 1-25, September.
    7. Lorraine Whitmarsh & Dimitrios Xenias & Christopher R. Jones, 2019. "Framing effects on public support for carbon capture and storage," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-10, December.
    8. Ganga Shreedhar & Susana Mourato, 2020. "Linking Human Destruction of Nature to COVID-19 Increases Support for Wildlife Conservation Policies," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 76(4), pages 963-999, August.
    9. Lawrence C. Hamilton, 2016. "Public Awareness of the Scientific Consensus on Climate," SAGE Open, , vol. 6(4), pages 21582440166, November.
    10. Tobias Greitemeyer, 2014. "I Am Right, You Are Wrong: How Biased Assimilation Increases the Perceived Gap between Believers and Skeptics of Violent Video Game Effects," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-7, April.
    11. Mei Tian & Yan Su & Xin Ru, 2016. "Perish or Publish in China: Pressures on Young Chinese Scholars to Publish in Internationally Indexed Journals," Publications, MDPI, vol. 4(2), pages 1-16, April.
    12. Pierre J C Chuard & Milan Vrtílek & Megan L Head & Michael D Jennions, 2019. "Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?," PLOS Biology, Public Library of Science, vol. 17(1), pages 1-7, January.
    13. Bruce Tranter & Zlatko Skrbis, 2014. "Political and Social Divisions over Climate Change among Young Queenslanders," Environment and Planning A, , vol. 46(7), pages 1638-1651, July.
    14. Charles A. Ogunbode & Rouven Doran & Gisela Böhm, 2020. "Exposure to the IPCC special report on 1.5 °C global warming is linked to perceived threat and increased concern about climate change," Climatic Change, Springer, vol. 158(3), pages 361-375, February.
    15. van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2018. "P-uniform," MetaArXiv zqjr9, Center for Open Science.
    16. Mary Guillard & Ghozlane Fleury-Bahi & Oscar Navarro, 2021. "Encouraging Individuals to Adapt to Climate Change: Relations between Coping Strategies and Psychological Distance," Sustainability, MDPI, vol. 13(2), pages 1-15, January.
    17. Marko Hofmann & Silja Meyer-Nieberg, 2018. "Time to dispense with the p-value in OR?," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 26(1), pages 193-214, March.
    18. Hladchenko, Myroslava & Moed, Henk F., 2021. "The effect of publication traditions and requirements in research assessment and funding policies upon the use of national journals in 28 post-socialist countries," Journal of Informetrics, Elsevier, vol. 15(4).
    19. Walter Leal Filho & Robert Stojanov & Franziska Wolf & Newton R. Matandirotya & Christian Ploberger & Desalegn Y. Ayal & Fardous Mohammad Safiul Azam & Tareq Mohammed Ali AL-Ahdal & Rebecca Sarku & No, 2022. "Assessing Uncertainties in Climate Change Adaptation and Land Management," Land, MDPI, vol. 11(12), pages 1-15, December.
    20. Grant R. McDermott, 2021. "Skeptic priors and climate consensus," Climatic Change, Springer, vol. 166(1), pages 1-23, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:140:y:2017:i:3:d:10.1007_s10584-016-1880-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.