IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/a8gu5.html

Addressing selective reporting of experiments – the case for predefined exclusion criteria

Author

Listed:
  • Neves, Kleber
  • Amaral, Olavo Bohrer

Abstract

Articles describing experimental data in the life sciences are meant to tell a clear story to the reader. This means that not every experimental attempt ends up published, as failed experiments and uninformative data are typically filtered out by researchers. Freedom to exclude data from an article, however, can lead to reporting bias when exclusion decisions are made after results are in. We discuss how to balance clarity and thoroughness in biomedical research reporting, and suggest that predefined criteria for experimental validity might help in solving this conflict.

Suggested Citation

  • Neves, Kleber & Amaral, Olavo Bohrer, 2019. "Addressing selective reporting of experiments – the case for predefined exclusion criteria," MetaArXiv a8gu5, Center for Open Science.
    • Handle: RePEc:osf:metaar:a8gu5
    DOI: 10.31219/osf.io/a8gu5
    as

    Download full text from publisher

    File URL: https://osf.io/download/5dfa860c0236b8000c7338c6/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/a8gu5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
    2. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    3. Luke Holman & Megan L Head & Robert Lanfear & Michael D Jennions, 2015. "Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording," PLOS Biology, Public Library of Science, vol. 13(7), pages 1-12, July.
    4. Carol Kilkenny & William J Browne & Innes C Cuthill & Michael Emerson & Douglas G Altman, 2010. "Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research," PLOS Biology, Public Library of Science, vol. 8(6), pages 1-5, June.
    5. Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    6. Jeffrey S. Mogil & Malcolm R. Macleod, 2017. "No publication without confirmation," Nature, Nature, vol. 542(7642), pages 409-411, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gillian L Currie & Helena N Angel-Scott & Lesley Colvin & Fala Cramond & Kaitlyn Hair & Laila Khandoker & Jing Liao & Malcolm Macleod & Sarah K McCann & Rosie Morland & Nicki Sherratt & Robert Stewart, 2019. "Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-34, May.
    2. Heesen, Remco & Bright, Liam Kofi, 2025. "Publication bias is bad for science if not necessarily scientists," LSE Research Online Documents on Economics 127420, London School of Economics and Political Science, LSE Library.
    3. Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
    4. Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
    5. Hannah Fraser & Tim Parker & Shinichi Nakagawa & Ashley Barnett & Fiona Fidler, 2018. "Questionable research practices in ecology and evolution," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-16, July.
    6. repec:plo:pone00:0193758 is not listed on IDEAS
    7. Pierre J C Chuard & Milan Vrtílek & Megan L Head & Michael D Jennions, 2019. "Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?," PLOS Biology, Public Library of Science, vol. 17(1), pages 1-7, January.
    8. Emanuele Rinninella & Marco Cintoni & Pauline Raoul & Vincenzina Mora & Antonio Gasbarrini & Maria Cristina Mele, 2021. "Impact of Food Additive Titanium Dioxide on Gut Microbiota Composition, Microbiota-Associated Functions, and Gut Barrier: A Systematic Review of In Vivo Animal Studies," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    9. Airi Jo-Watanabe & Toshiki Inaba & Takahiro Osada & Ryota Hashimoto & Tomohiro Nishizawa & Toshiaki Okuno & Sayoko Ihara & Kazushige Touhara & Nobutaka Hattori & Masatsugu Oh-Hora & Osamu Nureki & Tak, 2024. "Bicarbonate signalling via G protein-coupled receptor regulates ischaemia-reperfusion injury," Nature Communications, Nature, vol. 15(1), pages 1-18, December.
    10. Matthews, Michael J. & Anglin, Aaron H. & Drover, Will & Wolfe, Marcus T., 2024. "Just a number? Using artificial intelligence to explore perceived founder age in entrepreneurial fundraising," Journal of Business Venturing, Elsevier, vol. 39(1).
    11. Ashima Sakhuja & Ritobrata Bhattacharyya & Yash Tushar Katakia & Shyam Kumar Ramakrishnan & Srinjoy Chakraborty & Hariharan Jayakumar & Shailesh Mani Tripathi & Niyati Pandya Thakkar & Sumukh Thakar &, 2025. "S-nitrosylation of EZH2 alters PRC2 assembly, methyltransferase activity, and EZH2 stability to maintain endothelial homeostasis," Nature Communications, Nature, vol. 16(1), pages 1-23, December.
    12. Daniele Fanelli & Wolfgang Glänzel, 2013. "Bibliometric Evidence for a Hierarchy of the Sciences," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-11, June.
    13. repec:osf:osfxxx:f7a8c_v1 is not listed on IDEAS
    14. Ina Jahreis & Pablo Bascuñana & Tobias L Ross & Jens P Bankstahl & Marion Bankstahl, 2021. "Choice of anesthesia and data analysis method strongly increases sensitivity of 18F-FDG PET imaging during experimental epileptogenesis," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-20, November.
    15. James C. Ryan & Syed A. A. Tipu, 2016. "An Empirical Alternative to Sidani and Thornberry’s (2009) ‘Current Arab Work Ethic’: Examining the Multidimensional Work Ethic Profile in an Arab Context," Journal of Business Ethics, Springer, vol. 135(1), pages 177-198, April.
    16. repec:plo:pone00:0215221 is not listed on IDEAS
    17. Carlijn R Hooijmans & Rob B M de Vries & Maroeska M Rovers & Hein G Gooszen & Merel Ritskes-Hoitinga, 2012. "The Effects of Probiotic Supplementation on Experimental Acute Pancreatitis: A Systematic Review and Meta-Analysis," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-12, November.
    18. Mattia Prosperi & Jiang Bian & Iain E. Buchan & James S. Koopman & Matthew Sperrin & Mo Wang, 2019. "Raiders of the lost HARK: a reproducible inference framework for big data science," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 5(1), pages 1-12, December.
    19. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    20. Tracey L Weissgerber, 2021. "Learning from the past to develop data analysis curricula for the future," PLOS Biology, Public Library of Science, vol. 19(7), pages 1-3, July.
    21. M Polyakova & M L Schroeter & B M Elzinga & S Holiga & P Schoenknecht & E R de Kloet & M L Molendijk, 2015. "Brain-Derived Neurotrophic Factor and Antidepressive Effect of Electroconvulsive Therapy: Systematic Review and Meta-Analyses of the Preclinical and Clinical Literature," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-18, November.
    22. Nathaniel Poor, 2020. "Open-Source’s Inspirations for Computational Social Science: Lessons from a Failed Analysis," Media and Communication, Cogitatio Press, vol. 8(3), pages 231-238.
    23. Cantone, Giulio Giacomo, 2023. "The multiversal methodology as a remedy of the replication crisis," MetaArXiv kuhmz, Center for Open Science.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:a8gu5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.