IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/3002129.html
   My bibliography  Save this article

Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies

Author

Listed:
  • Benjamin Phillips
  • Timo N Haschler
  • Natasha A Karp

Abstract

In recent years, there has been a strong drive to improve the inclusion of animals of both sexes in the design of in vivo research studies, driven by a need to increase sex representation in fundamental biology and drug development. This has resulted in inclusion mandates by funding bodies and journals, alongside numerous published manuscripts highlighting the issue and providing guidance to scientists. However, progress is slow and barriers to the routine use of both sexes remain. A frequent, major concern is the perceived need for a higher overall sample size to achieve an equivalent level of statistical power, which would result in an increased ethical and resource burden. This perception arises from either the belief that sex inclusion will increase variability in the data (either through a baseline difference or a treatment effect that depends on sex), thus reducing the sensitivity of statistical tests, or from misapprehensions about the correct way to analyse the data, including disaggregation or pooling by sex. Here, we conduct an in-depth examination of the consequences of including both sexes on statistical power. We performed simulations by constructing artificial datasets that encompass a range of outcomes that may occur in studies studying a treatment effect in the context of both sexes. This includes both baseline sex differences and situations in which the size of the treatment effect depends on sex in both the same and opposite directions. The data were then analysed using either a factorial analysis approach, which is appropriate for the design, or a t test approach following pooling or disaggregation of the data, which are common but erroneous strategies. The results demonstrate that there is no loss of power to detect treatment effects when splitting the sample size across sexes in most scenarios, providing that the data are analysed using an appropriate factorial analysis method (e.g., two-way ANOVA). In the rare situations where power is lost, the benefit of understanding the role of sex outweighs the power considerations. Additionally, use of the inappropriate analysis pipelines results in a loss of statistical power. Therefore, we recommend analysing data collected from both sexes using factorial analysis and splitting the sample size across male and female mice as a standard strategy.Many scientists believe that studying both sexes by default requires an increase in the number of animals needed; is this correct? This study provides an in-depth examination of the topic through simulations to assess the reality and truly answer the question.

Suggested Citation

  • Benjamin Phillips & Timo N Haschler & Natasha A Karp, 2023. "Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies," PLOS Biology, Public Library of Science, vol. 21(6), pages 1-14, June.
  • Handle: RePEc:plo:pbio00:3002129
    DOI: 10.1371/journal.pbio.3002129
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002129
    Download Restriction: no

    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.3002129&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pbio.3002129?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. R. Douglas Fields, 2014. "NIH policy: Mandate goes too far," Nature, Nature, vol. 510(7505), pages 340-340, June.
    2. Valentin Amrhein & Sander Greenland & Blake McShane, 2019. "Scientists rise up against statistical significance," Nature, Nature, vol. 567(7748), pages 305-307, March.
    3. Rebecca K. Rechlin & Tallinn F. L. Splinter & Travis E. Hodges & Arianne Y. Albert & Liisa A. M. Galea, 2022. "An analysis of neuroscience and psychiatry papers published from 2009 and 2019 outlines opportunities for increasing discovery of sex differences," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Natasha A. Karp & Manuel Berdoy & Kelly Gray & Lilian Hunt & Maggy Jennings & Angela Kerton & Matt Leach & Jordi L. Tremoleda & Jon Gledhill & Esther J. Pearl & Nathalie Percie du Sert & Benjamin Phil, 2025. "The Sex Inclusive Research Framework to address sex bias in preclinical research proposals," Nature Communications, Nature, vol. 16(1), pages 1-6, December.
    2. Szymon M Drobniak & Malgorzata Lagisz & Yefeng Yang & Shinichi Nakagawa, 2024. "Realism and robustness require increased sample size when studying both sexes," PLOS Biology, Public Library of Science, vol. 22(4), pages 1-6, April.
    3. Benjamin Phillips & Timo N Haschler & Natasha A Karp, 2024. "A response to “Realism and robustness require increased sample size when studying both sexes”," PLOS Biology, Public Library of Science, vol. 22(4), pages 1-2, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mia Papasideris & Scott T Leatherdale & Kate Battista & Peter A Hall, 2021. "An examination of the prospective association between physical activity and academic achievement in youth at the population level," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-15, June.
    2. Surhan Cam & Serap Palaz, 2023. "Mutual interests management with a purposive approach: Evidence from the Turkish shipyards for an amorphous impact model between (subjective) well‐being and performance," Industrial Relations Journal, Wiley Blackwell, vol. 54(1), pages 40-70, January.
    3. Siddharth Sareen & Andrea Saltelli & Kjetil Rommetveit, 2020. "Ethics of quantification: illumination, obfuscation and performative legitimation," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-5, December.
    4. Tracy, Elizabeth M. & Billingsley, Joseph & Pollack, Jeffrey M. & Barber, Dennis & Beorchia, Ace & Carr, Jon C. & Gonzalez, Gabe & Harris, Michael L. & Michaelis, Timothy L. & Morrow, Grayson & Philli, 2021. "A behavioral insights approach to recruiting entrepreneurs for an academic study during the COVID-19 pandemic," Journal of Business Venturing Insights, Elsevier, vol. 16(C).
    5. Lars Mewes & Leonie Tuitjer & Peter Dirksmeier, 2024. "Exploring the variances of climate change opinions in Germany at a fine-grained local scale," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    6. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    7. Kristin B. Dobbin & Amanda L. Fencl & Gregory Pierce & Melissa Beresford & Silvia Gonzalez & Wendy Jepson, 2023. "Understanding perceived climate risks to household water supply and their implications for adaptation: evidence from California," Climatic Change, Springer, vol. 176(4), pages 1-20, April.
    8. Abbas, Sadia & Adapa, Sujana & Sheridan, Alison & Azeem, Muhammad Masood, 2022. "Informal competition and firm level innovation in South Asia: The moderating role of innovation time off and R&D intensity," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    9. Elizabeth P.D. Koselka & Lucy C. Weidner & Arseniy Minasov & Marc G. Berman & William R. Leonard & Marianne V. Santoso & Junia N. de Brito & Zachary C. Pope & Mark A. Pereira & Teresa H. Horton, 2019. "Walking Green: Developing an Evidence Base for Nature Prescriptions," IJERPH, MDPI, vol. 16(22), pages 1-18, November.
    10. Piotr Skórka & Beata Grzywacz & Dawid Moroń & Magdalena Lenda, 2020. "The macroecology of the COVID-19 pandemic in the Anthropocene," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-17, July.
    11. Susan Zief & John Deke & Paul Burkander & Andrew Langan & Subuhi Asheer, "undated". "Impacts of a Home Visiting Program Enhanced with Content on Healthy Birth Spacing," Mathematica Policy Research Reports 230745c77416490bb743dae6f, Mathematica Policy Research.
    12. Le, Kien & Nguyen, My, 2020. "Shedding light on maternal education and child health in developing countries," World Development, Elsevier, vol. 133(C).
    13. Helmut Wasserbacher & Martin Spindler, 2024. "Credit Ratings: Heterogeneous Effect on Capital Structure," Papers 2406.18936, arXiv.org.
    14. Santini, Ziggi Ivan & Thygesen, Lau Caspar & Koyanagi, Ai & Stewart-Brown, Sarah & Meilstrup, Charlotte & Nielsen, Line & Olsen, Kim Rose & Birkjær, Michael & McDaid, David & Koushede, Vibeke & Ekholm, 2022. "Economics of mental wellbeing: a prospective study estimating associated productivity costs due to sickness absence from the workplace in Denmark," LSE Research Online Documents on Economics 116690, London School of Economics and Political Science, LSE Library.
    15. Chenxi Zhao & Chenglei Zhao & Minmin Zhao & Lin Wang & Jiawei Guo & Longhai Zhang & Yunfeng Li & Yuliang Sun & Ling Zhang & Zheng’ao Li & Wenfei Zhu, 2022. "Effect of Exergame Training on Working Memory and Executive Function in Older Adults," Sustainability, MDPI, vol. 14(17), pages 1-11, August.
    16. Tim Bothe & Josephine Jacob & Christoph Kröger & Jochen Walker, 2020. "How expensive are post-traumatic stress disorders? Estimating incremental health care and economic costs on anonymised claims data," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(6), pages 917-930, August.
    17. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    18. Ashley A Leech & Cindy L Christiansen & Benjamin P Linas & Donna M Jacobsen & Isabel Morin & Mari-Lynn Drainoni, 2020. "Healthcare practitioner experiences and willingness to prescribe pre-exposure prophylaxis in the US," PLOS ONE, Public Library of Science, vol. 15(9), pages 1-12, September.
    19. Hans Van Remoortel & Hans Scheers & Emmy De Buck & Winne Haenen & Philippe Vandekerckhove, 2020. "Prediction modelling studies for medical usage rates in mass gatherings: A systematic review," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-20, June.
    20. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:3002129. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.