IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/1002331.html
   My bibliography  Save this article

Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke

Author

Listed:
  • Constance Holman
  • Sophie K Piper
  • Ulrike Grittner
  • Andreas Antonios Diamantaras
  • Jonathan Kimmelman
  • Bob Siegerink
  • Ulrich Dirnagl

Abstract

Given small sample sizes, loss of animals in preclinical experiments can dramatically alter results. However, effects of attrition on distortion of results are unknown. We used a simulation study to analyze the effects of random and biased attrition. As expected, random loss of samples decreased statistical power, but biased removal, including that of outliers, dramatically increased probability of false positive results. Next, we performed a meta-analysis of animal reporting and attrition in stroke and cancer. Most papers did not adequately report attrition, and extrapolating from the results of the simulation data, we suggest that their effect sizes were likely overestimated.Using a combination of simulation and meta-analysis of stroke and cancer studies, this article highlights the potentially seriously misleading consequences of failing to report the loss of animals in preclinical studies.

Suggested Citation

  • Constance Holman & Sophie K Piper & Ulrike Grittner & Andreas Antonios Diamantaras & Jonathan Kimmelman & Bob Siegerink & Ulrich Dirnagl, 2016. "Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-12, January.
    • Handle: RePEc:plo:pbio00:1002331
    DOI: 10.1371/journal.pbio.1002331
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002331
    Download Restriction: no
    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.1002331&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pbio.1002331?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Beverly S Muhlhausler & Frank H Bloomfield & Matthew W Gillman, 2013. "Whole Animal Experiments Should Be More Like Human Randomized Controlled Trials," PLOS Biology, Public Library of Science, vol. 11(2), pages 1-6, February.
    2. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    3. Konstantinos K Tsilidis & Orestis A Panagiotou & Emily S Sena & Eleni Aretouli & Evangelos Evangelou & David W Howells & Rustam Al-Shahi Salman & Malcolm R Macleod & John P A Ioannidis, 2013. "Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases," PLOS Biology, Public Library of Science, vol. 11(7), pages 1-10, July.
    4. Matthew James Shun-Shin & Darrel P Francis, 2013. "Why Even More Clinical Research Studies May Be False: Effect of Asymmetrical Handling of Clinically Unexpected Values," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-6, June.
    5. H Bart van der Worp & David W Howells & Emily S Sena & Michelle J Porritt & Sarah Rewell & Victoria O'Collins & Malcolm R Macleod, 2010. "Can Animal Models of Disease Reliably Inform Human Studies?," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-8, March.
    6. Emily S Sena & H Bart van der Worp & Philip M W Bath & David W Howells & Malcolm R Macleod, 2010. "Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy," PLOS Biology, Public Library of Science, vol. 8(3), pages 1-8, March.
    7. Carol Kilkenny & William J Browne & Innes C Cuthill & Michael Emerson & Douglas G Altman, 2010. "Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research," PLOS Biology, Public Library of Science, vol. 8(6), pages 1-5, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gillian L Currie & Helena N Angel-Scott & Lesley Colvin & Fala Cramond & Kaitlyn Hair & Laila Khandoker & Jing Liao & Malcolm Macleod & Sarah K McCann & Rosie Morland & Nicki Sherratt & Robert Stewart, 2019. "Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-34, May.
    2. Goeschl, Timo & Heyen, Daniel, 2016. "Precision requirements in pesticide risk assessments: Contrasting value-of-information recommendations with the regulatory practice in the EU," Working Papers 0607, University of Heidelberg, Department of Economics.
    3. Tracey L Weissgerber, 2021. "Learning from the past to develop data analysis curricula for the future," PLOS Biology, Public Library of Science, vol. 19(7), pages 1-3, July.
    4. Markus Lehmkuhl & Nikolai Promies, 2020. "Frequency distribution of journalistic attention for scientific studies and scientific sources: An input–output analysis," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-20, November.
    5. Matthias Steinfath & Silvia Vogl & Norman Violet & Franziska Schwarz & Hans Mielke & Thomas Selhorst & Matthias Greiner & Gilbert Schönfelder, 2018. "Simple changes of individual studies can improve the reproducibility of the biomedical scientific process as a whole," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-20, September.
    6. Stavroula Kousta & Christine Ferguson & Emma Ganley, 2016. "Meta-Research: Broadening the Scope of PLOS Biology," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-2, January.
    7. Neves, Kleber & Amaral, Olavo Bohrer, 2019. "Addressing selective reporting of experiments – the case for predefined exclusion criteria," MetaArXiv a8gu5, Center for Open Science.
    8. Tracey L Weissgerber & Vesna D Garovic & Marko Savic & Stacey J Winham & Natasa M Milic, 2016. "From Static to Interactive: Transforming Data Visualization to Improve Transparency," PLOS Biology, Public Library of Science, vol. 14(6), pages 1-8, June.
    9. Tracey L Weissgerber & Vesna D Garovic & Jelena S Milin-Lazovic & Stacey J Winham & Zoran Obradovic & Jerome P Trzeciakowski & Natasa M Milic, 2016. "Reinventing Biostatistics Education for Basic Scientists," PLOS Biology, Public Library of Science, vol. 14(4), pages 1-12, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Baker & Katie Lidster & Ana Sottomayor & Sandra Amor, 2014. "Two Years Later: Journals Are Not Yet Enforcing the ARRIVE Guidelines on Reporting Standards for Pre-Clinical Animal Studies," PLOS Biology, Public Library of Science, vol. 12(1), pages 1-6, January.
    2. Susanne Wieschowski & Diego S Silva & Daniel Strech, 2016. "Animal Study Registries: Results from a Stakeholder Analysis on Potential Strengths, Weaknesses, Facilitators, and Barriers," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-12, November.
    3. Konstantinos K Tsilidis & Orestis A Panagiotou & Emily S Sena & Eleni Aretouli & Evangelos Evangelou & David W Howells & Rustam Al-Shahi Salman & Malcolm R Macleod & John P A Ioannidis, 2013. "Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases," PLOS Biology, Public Library of Science, vol. 11(7), pages 1-10, July.
    4. Gillian L Currie & Helena N Angel-Scott & Lesley Colvin & Fala Cramond & Kaitlyn Hair & Laila Khandoker & Jing Liao & Malcolm Macleod & Sarah K McCann & Rosie Morland & Nicki Sherratt & Robert Stewart, 2019. "Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-34, May.
    5. Kimberley E Wever & Carlijn R Hooijmans & Niels P Riksen & Thomas B Sterenborg & Emily S Sena & Merel Ritskes-Hoitinga & Michiel C Warlé, 2015. "Determinants of the Efficacy of Cardiac Ischemic Preconditioning: A Systematic Review and Meta-Analysis of Animal Studies," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-17, November.
    6. Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
    7. Stephen Fox, 2018. "Irresponsible Research and Innovation? Applying Findings from Neuroscience to Analysis of Unsustainable Hype Cycles," Sustainability, MDPI, vol. 10(10), pages 1-16, September.
    8. Stephan B Bruns & John P A Ioannidis, 2016. "p-Curve and p-Hacking in Observational Research," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-13, February.
    9. Nathalie Percie du Sert & Viki Hurst & Amrita Ahluwalia & Sabina Alam & Marc T Avey & Monya Baker & William J Browne & Alejandra Clark & Innes C Cuthill & Ulrich Dirnagl & Michael Emerson & Paul Garne, 2020. "The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-12, July.
    10. Vivian Leung & Frédérik Rousseau-Blass & Guy Beauchamp & Daniel S J Pang, 2018. "ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal Research: Reporting of in vivo Experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesi," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-13, May.
    11. Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
    12. Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
    13. Jenny T van der Steen & Cornelis A van den Bogert & Mirjam C van Soest-Poortvliet & Soulmaz Fazeli Farsani & René H J Otten & Gerben ter Riet & Lex M Bouter, 2018. "Determinants of selective reporting: A taxonomy based on content analysis of a random selection of the literature," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-15, February.
    14. Jennifer A Hirst & Jeremy Howick & Jeffrey K Aronson & Nia Roberts & Rafael Perera & Constantinos Koshiaris & Carl Heneghan, 2014. "The Need for Randomization in Animal Trials: An Overview of Systematic Reviews," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-11, June.
    15. Tugce Yildizoglu & Jan-Marek Weislogel & Farhan Mohammad & Edwin S-Y Chan & Pryseley N Assam & Adam Claridge-Chang, 2015. "Estimating Information Processing in a Memory System: The Utility of Meta-analytic Methods for Genetics," PLOS Genetics, Public Library of Science, vol. 11(12), pages 1-27, December.
    16. Carol Kilkenny & William J Browne & Innes C Cuthill & Michael Emerson & Douglas G Altman, 2010. "Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research," PLOS Biology, Public Library of Science, vol. 8(6), pages 1-5, June.
    17. Hristo Todorov & Emily Searle-White & Susanne Gerber, 2020. "Applying univariate vs. multivariate statistics to investigate therapeutic efficacy in (pre)clinical trials: A Monte Carlo simulation study on the example of a controlled preclinical neurotrauma trial," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-20, March.
    18. Ana Antonic & Emily S Sena & Jennifer S Lees & Taryn E Wills & Peta Skeers & Peter E Batchelor & Malcolm R Macleod & David W Howells, 2013. "Stem Cell Transplantation in Traumatic Spinal Cord Injury: A Systematic Review and Meta-Analysis of Animal Studies," PLOS Biology, Public Library of Science, vol. 11(12), pages 1-14, December.
    19. Julie E. Goodman & Catherine Petito Boyce & Sonja N. Sax & Leslie A. Beyer & Robyn L. Prueitt, 2015. "Rethinking Meta‐Analysis: Applications for Air Pollution Data and Beyond," Risk Analysis, John Wiley & Sons, vol. 35(6), pages 1017-1039, June.
    20. Airi Jo-Watanabe & Toshiki Inaba & Takahiro Osada & Ryota Hashimoto & Tomohiro Nishizawa & Toshiaki Okuno & Sayoko Ihara & Kazushige Touhara & Nobutaka Hattori & Masatsugu Oh-Hora & Osamu Nureki & Tak, 2024. "Bicarbonate signalling via G protein-coupled receptor regulates ischaemia-reperfusion injury," Nature Communications, Nature, vol. 15(1), pages 1-18, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:1002331. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.