IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/a8gu5.html
   My bibliography  Save this paper

Addressing selective reporting of experiments – the case for predefined exclusion criteria

Author

Listed:
  • Neves, Kleber
  • Amaral, Olavo Bohrer

Abstract

Articles describing experimental data in the life sciences are meant to tell a clear story to the reader. This means that not every experimental attempt ends up published, as failed experiments and uninformative data are typically filtered out by researchers. Freedom to exclude data from an article, however, can lead to reporting bias when exclusion decisions are made after results are in. We discuss how to balance clarity and thoroughness in biomedical research reporting, and suggest that predefined criteria for experimental validity might help in solving this conflict.

Suggested Citation

  • Neves, Kleber & Amaral, Olavo Bohrer, 2019. "Addressing selective reporting of experiments – the case for predefined exclusion criteria," MetaArXiv a8gu5, Center for Open Science.
    • Handle: RePEc:osf:metaar:a8gu5
    DOI: 10.31219/osf.io/a8gu5
    as

    Download full text from publisher

    File URL: https://osf.io/download/5dfa860c0236b8000c7338c6/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/a8gu5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jeffrey S. Mogil & Malcolm R. Macleod, 2017. "No publication without confirmation," Nature, Nature, vol. 542(7642), pages 409-411, February.
    2. Carol Kilkenny & William J Browne & Innes C Cuthill & Michael Emerson & Douglas G Altman, 2010. "Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research," PLOS Biology, Public Library of Science, vol. 8(6), pages 1-5, June.
    3. Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    4. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    5. Luke Holman & Megan L Head & Robert Lanfear & Michael D Jennions, 2015. "Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording," PLOS Biology, Public Library of Science, vol. 13(7), pages 1-12, July.
    6. Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
    2. Hannah Fraser & Tim Parker & Shinichi Nakagawa & Ashley Barnett & Fiona Fidler, 2018. "Questionable research practices in ecology and evolution," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-16, July.
    3. Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
    4. Gillian L Currie & Helena N Angel-Scott & Lesley Colvin & Fala Cramond & Kaitlyn Hair & Laila Khandoker & Jing Liao & Malcolm Macleod & Sarah K McCann & Rosie Morland & Nicki Sherratt & Robert Stewart, 2019. "Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-34, May.
    5. repec:plo:pone00:0193758 is not listed on IDEAS
    6. repec:plo:pone00:0215221 is not listed on IDEAS
    7. Vigren, Andreas & Pyddoke, Roger, 2020. "The impact on bus ridership of passenger incentive contracts in public transport," Transportation Research Part A: Policy and Practice, Elsevier, vol. 135(C), pages 144-159.
    8. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    9. Robert J Warren II & Joshua R King & Charlene Tarsa & Brian Haas & Jeremy Henderson, 2017. "A systematic review of context bias in invasion biology," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-12, August.
    10. Jasper Brinkerink, 2023. "When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research," Entrepreneurship Theory and Practice, , vol. 47(2), pages 304-343, March.
    11. Julia Jerke & Antonia Velicu & Fabian Winter & Heiko Rauhut, 2025. "Publication bias in the social sciences since 1959: Application of a regression discontinuity framework," PLOS ONE, Public Library of Science, vol. 20(2), pages 1-27, February.
    12. Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
    13. Zhongwei Xu & Bingze Xu & Susanna L. Lundström & Àlex Moreno-Giró & Danxia Zhao & Myriam Martin & Erik Lönnblom & Qixing Li & Alexander Krämer & Changrong Ge & Lei Cheng & Bibo Liang & Dongmei Tong & , 2023. "A subset of type-II collagen-binding antibodies prevents experimental arthritis by inhibiting FCGR3 signaling in neutrophils," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    14. Christian Heise & Joshua M. Pearce, 2020. "From Open Access to Open Science: The Path From Scientific Reality to Open Scientific Communication," SAGE Open, , vol. 10(2), pages 21582440209, May.
    15. Vivian Leung & Frédérik Rousseau-Blass & Guy Beauchamp & Daniel S J Pang, 2018. "ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesi," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-13, May.
    16. repec:plo:pone00:0240719 is not listed on IDEAS
    17. Reed, W. Robert, 2019. "Takeaways from the special issue on The Practice of Replication," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 13, pages 1-11.
    18. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    19. Daiping Wang & Wolfgang Forstmeier & Mihai Valcu & Niels J Dingemanse & Martin Bulla & Christiaan Both & Renée A Duckworth & Lynna Marie Kiere & Patrik Karell & Tomáš Albrecht & Bart Kempenaers, 2019. "Scrutinizing assortative mating in birds," PLOS Biology, Public Library of Science, vol. 17(2), pages 1-20, February.
    20. Shuaijun Guo & Xiaoming Yu & Orkan Okan, 2020. "Moving Health Literacy Research and Practice towards a Vision of Equity, Precision and Transparency," IJERPH, MDPI, vol. 17(20), pages 1-14, October.
    21. Beverly S Muhlhausler & Frank H Bloomfield & Matthew W Gillman, 2013. "Whole Animal Experiments Should Be More Like Human Randomized Controlled Trials," PLOS Biology, Public Library of Science, vol. 11(2), pages 1-6, February.
    22. Dominika Ehrenbergerova & Josef Bajzik & Tomas Havranek, 2023. "When Does Monetary Policy Sway House Prices? A Meta-Analysis," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 538-573, June.
    23. Rubin, Mark, 2020. "Does preregistration improve the credibility of research findings?," MetaArXiv vgr89, Center for Open Science.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:a8gu5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.