IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0084896.html
   My bibliography  Save this article

Why Publishing Everything Is More Effective than Selective Publishing of Statistically Significant Results

Author

Listed:
  • Marcel A L M van Assen
  • Robbie C M van Aert
  • Michèle B Nuijten
  • Jelte M Wicherts

Abstract

Background: De Winter and Happee [1] examined whether science based on selective publishing of significant results may be effective in accurate estimation of population effects, and whether this is even more effective than a science in which all results are published (i.e., a science without publication bias). Based on their simulation study they concluded that “selective publishing yields a more accurate meta-analytic estimation of the true effect than publishing everything, (and that) publishing nonreplicable results while placing null results in the file drawer can be beneficial for the scientific collective” (p.4). Methods and Findings: Using their scenario with a small to medium population effect size, we show that publishing everything is more effective for the scientific collective than selective publishing of significant results. Additionally, we examined a scenario with a null effect, which provides a more dramatic illustration of the superiority of publishing everything over selective publishing. Conclusion: Publishing everything is more effective than only reporting significant outcomes.

Suggested Citation

  • Marcel A L M van Assen & Robbie C M van Aert & Michèle B Nuijten & Jelte M Wicherts, 2014. "Why Publishing Everything Is More Effective than Selective Publishing of Statistically Significant Results," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-5, January.
    • Handle: RePEc:plo:pone00:0084896
    DOI: 10.1371/journal.pone.0084896
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0084896
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0084896&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0084896?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniele Fanelli, 2010. "“Positive” Results Increase Down the Hierarchy of the Sciences," PLOS ONE, Public Library of Science, vol. 5(4), pages 1-10, April.
    2. Joost de Winter & Riender Happee, 2013. "Why Selective Publication of Statistically Significant Results Can Be Effective," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-7, June.
    3. Emily S Sena & H Bart van der Worp & Philip M W Bath & David W Howells & Malcolm R Macleod, 2010. "Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy," PLOS Biology, Public Library of Science, vol. 8(3), pages 1-8, March.
    4. Gerber, Alan S. & Green, Donald P. & Nickerson, David, 2001. "Testing for Publication Bias in Political Science," Political Analysis, Cambridge University Press, vol. 9(4), pages 385-392, January.
    5. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marie Juanchich & Miroslav Sirota, 2016. "How to improve people's interpretation of probabilities of precipitation," Journal of Risk Research, Taylor & Francis Journals, vol. 19(3), pages 388-404, March.
    2. Leonid Tiokhin & Minhua Yan & Thomas J. H. Morgan, 2021. "Competition for priority harms the reliability of science, but reforms can help," Nature Human Behaviour, Nature, vol. 5(7), pages 857-867, July.
    3. Dwight C. K. Tse & Jeanne Nakamura & Mihaly Csikszentmihalyi, 2022. "Flow Experiences Across Adulthood: Preliminary Findings on the Continuity Hypothesis," Journal of Happiness Studies, Springer, vol. 23(6), pages 2517-2540, August.
    4. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    5. Furukawa, Chishio, 2019. "Publication Bias under Aggregation Frictions: Theory, Evidence, and a New Correction Method," EconStor Preprints 194798, ZBW - Leibniz Information Centre for Economics.
    6. Chris Hartgerink, 2019. "Verified, Shared, Modular, and Provenance Based Research Communication with the Dat Protocol," Publications, MDPI, vol. 7(2), pages 1-19, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniele Fanelli & Rodrigo Costas & Vincent Larivière, 2015. "Misconduct Policies, Academic Culture and Career Stage, Not Gender or Pressures to Publish, Affect Scientific Integrity," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-18, June.
    2. van Aert, Robbie Cornelis Maria, 2018. "Dissertation R.C.M. van Aert," MetaArXiv eqhjd, Center for Open Science.
    3. Piotr Bialowolski & Dorota Weziak-Bialowolska & Eileen McNeely, 2021. "The Role of Financial Fragility and Financial Control for Well-Being," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 155(3), pages 1137-1157, June.
    4. Fabo, Brian & Jančoková, Martina & Kempf, Elisabeth & Pástor, Ľuboš, 2024. "Fifty shades of QE: Robust evidence," Journal of Banking & Finance, Elsevier, vol. 159(C).
    5. Karin Langenkamp & Bodo Rödel & Kerstin Taufenbach & Meike Weiland, 2018. "Open Access in Vocational Education and Training Research," Publications, MDPI, vol. 6(3), pages 1-12, July.
    6. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    7. Vitor Azevedo & Christopher Hoegner, 2023. "Enhancing stock market anomalies with machine learning," Review of Quantitative Finance and Accounting, Springer, vol. 60(1), pages 195-230, January.
    8. Brian Fabo & Martina Jancokova & Elisabeth Kempf & Lubos Pastor, 2020. "Fifty Shades of QE: Conflicts of Interest in Economic Research," Working Papers 2020-128, Becker Friedman Institute for Research In Economics.
    9. Christian Heise & Joshua M. Pearce, 2020. "From Open Access to Open Science: The Path From Scientific Reality to Open Scientific Communication," SAGE Open, , vol. 10(2), pages 21582440209, May.
    10. Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
    11. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    12. Lisa Bero, 2018. "Meta-research matters: Meta-spin cycles, the blindness of bias, and rebuilding trust," PLOS Biology, Public Library of Science, vol. 16(4), pages 1-4, April.
    13. Bobtcheff, Catherine & Mariotti, Thomas & Levy, Raphaël, 2021. "Negative results in science: Blessing or (winner’s) curse," TSE Working Papers 21-1202, Toulouse School of Economics (TSE).
    14. Mei Tian & Yan Su & Xin Ru, 2016. "Perish or Publish in China: Pressures on Young Chinese Scholars to Publish in Internationally Indexed Journals," Publications, MDPI, vol. 4(2), pages 1-16, April.
    15. Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    16. Weziak-Bialowolska, Dorota & Lee, Matthew T. & Cowden, Richard G. & Bialowolski, Piotr & Chen, Ying & VanderWeele, Tyler J. & McNeely, Eileen, 2023. "Psychological caring climate at work, mental health, well-being, and work-related outcomes: Evidence from a longitudinal study and health insurance data," Social Science & Medicine, Elsevier, vol. 323(C).
    17. Pierre J C Chuard & Milan Vrtílek & Megan L Head & Michael D Jennions, 2019. "Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?," PLOS Biology, Public Library of Science, vol. 17(1), pages 1-7, January.
    18. Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
    19. Stanley, T. D. & Doucouliagos, Hristos, 2013. "Better than random: weighted least squares meta-regression analysis," Working Papers eco_2013_2, Deakin University, Department of Economics.
    20. Jakub Bijak, 2019. "Editorial: P-values, theory, replicability, and rigour," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 41(32), pages 949-952.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0084896. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.