IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/3002129.html
   My bibliography  Save this article

Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies

Author

Listed:
  • Benjamin Phillips
  • Timo N Haschler
  • Natasha A Karp

Abstract

In recent years, there has been a strong drive to improve the inclusion of animals of both sexes in the design of in vivo research studies, driven by a need to increase sex representation in fundamental biology and drug development. This has resulted in inclusion mandates by funding bodies and journals, alongside numerous published manuscripts highlighting the issue and providing guidance to scientists. However, progress is slow and barriers to the routine use of both sexes remain. A frequent, major concern is the perceived need for a higher overall sample size to achieve an equivalent level of statistical power, which would result in an increased ethical and resource burden. This perception arises from either the belief that sex inclusion will increase variability in the data (either through a baseline difference or a treatment effect that depends on sex), thus reducing the sensitivity of statistical tests, or from misapprehensions about the correct way to analyse the data, including disaggregation or pooling by sex. Here, we conduct an in-depth examination of the consequences of including both sexes on statistical power. We performed simulations by constructing artificial datasets that encompass a range of outcomes that may occur in studies studying a treatment effect in the context of both sexes. This includes both baseline sex differences and situations in which the size of the treatment effect depends on sex in both the same and opposite directions. The data were then analysed using either a factorial analysis approach, which is appropriate for the design, or a t test approach following pooling or disaggregation of the data, which are common but erroneous strategies. The results demonstrate that there is no loss of power to detect treatment effects when splitting the sample size across sexes in most scenarios, providing that the data are analysed using an appropriate factorial analysis method (e.g., two-way ANOVA). In the rare situations where power is lost, the benefit of understanding the role of sex outweighs the power considerations. Additionally, use of the inappropriate analysis pipelines results in a loss of statistical power. Therefore, we recommend analysing data collected from both sexes using factorial analysis and splitting the sample size across male and female mice as a standard strategy.Many scientists believe that studying both sexes by default requires an increase in the number of animals needed; is this correct? This study provides an in-depth examination of the topic through simulations to assess the reality and truly answer the question.

Suggested Citation

  • Benjamin Phillips & Timo N Haschler & Natasha A Karp, 2023. "Statistical simulations show that scientists need not increase overall sample size by default when including both sexes in in vivo studies," PLOS Biology, Public Library of Science, vol. 21(6), pages 1-14, June.
  • Handle: RePEc:plo:pbio00:3002129
    DOI: 10.1371/journal.pbio.3002129
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002129
    Download Restriction: no

    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.3002129&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pbio.3002129?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Rebecca K. Rechlin & Tallinn F. L. Splinter & Travis E. Hodges & Arianne Y. Albert & Liisa A. M. Galea, 2022. "An analysis of neuroscience and psychiatry papers published from 2009 and 2019 outlines opportunities for increasing discovery of sex differences," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    2. R. Douglas Fields, 2014. "NIH policy: Mandate goes too far," Nature, Nature, vol. 510(7505), pages 340-340, June.
    3. Valentin Amrhein & Sander Greenland & Blake McShane, 2019. "Scientists rise up against statistical significance," Nature, Nature, vol. 567(7748), pages 305-307, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Szymon M Drobniak & Malgorzata Lagisz & Yefeng Yang & Shinichi Nakagawa, 2024. "Realism and robustness require increased sample size when studying both sexes," PLOS Biology, Public Library of Science, vol. 22(4), pages 1-6, April.
    2. Benjamin Phillips & Timo N Haschler & Natasha A Karp, 2024. "A response to “Realism and robustness require increased sample size when studying both sexes”," PLOS Biology, Public Library of Science, vol. 22(4), pages 1-2, April.
    3. Natasha A. Karp & Manuel Berdoy & Kelly Gray & Lilian Hunt & Maggy Jennings & Angela Kerton & Matt Leach & Jordi L. Tremoleda & Jon Gledhill & Esther J. Pearl & Nathalie Percie du Sert & Benjamin Phil, 2025. "The Sex Inclusive Research Framework to address sex bias in preclinical research proposals," Nature Communications, Nature, vol. 16(1), pages 1-6, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mia Papasideris & Scott T Leatherdale & Kate Battista & Peter A Hall, 2021. "An examination of the prospective association between physical activity and academic achievement in youth at the population level," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-15, June.
    2. Surhan Cam & Serap Palaz, 2023. "Mutual interests management with a purposive approach: Evidence from the Turkish shipyards for an amorphous impact model between (subjective) well‐being and performance," Industrial Relations Journal, Wiley Blackwell, vol. 54(1), pages 40-70, January.
    3. Michael Essman & Lindsey Smith Taillie & Tamryn Frank & Shu Wen Ng & Barry M Popkin & Elizabeth C Swart, 2021. "Taxed and untaxed beverage intake by South African young adults after a national sugar-sweetened beverage tax: A before-and-after study," PLOS Medicine, Public Library of Science, vol. 18(5), pages 1-17, May.
    4. Siddharth Sareen & Andrea Saltelli & Kjetil Rommetveit, 2020. "Ethics of quantification: illumination, obfuscation and performative legitimation," Palgrave Communications, Palgrave Macmillan, vol. 6(1), pages 1-5, December.
    5. Francesco De Pretis & Barbara Osimani, 2019. "New Insights in Computational Methods for Pharmacovigilance: E-Synthesis , a Bayesian Framework for Causal Assessment," IJERPH, MDPI, vol. 16(12), pages 1-19, June.
    6. Tracy, Elizabeth M. & Billingsley, Joseph & Pollack, Jeffrey M. & Barber, Dennis & Beorchia, Ace & Carr, Jon C. & Gonzalez, Gabe & Harris, Michael L. & Michaelis, Timothy L. & Morrow, Grayson & Philli, 2021. "A behavioral insights approach to recruiting entrepreneurs for an academic study during the COVID-19 pandemic," Journal of Business Venturing Insights, Elsevier, vol. 16(C).
    7. Lars Mewes & Leonie Tuitjer & Peter Dirksmeier, 2024. "Exploring the variances of climate change opinions in Germany at a fine-grained local scale," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    8. Azeem, Muhammad Masood & Sheridan, Alison & Adapa, Sujana, 2022. "Women to women: Enabling innovation and firm performance in developing countries," Emerging Markets Review, Elsevier, vol. 51(PA).
    9. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    10. Kristin B. Dobbin & Amanda L. Fencl & Gregory Pierce & Melissa Beresford & Silvia Gonzalez & Wendy Jepson, 2023. "Understanding perceived climate risks to household water supply and their implications for adaptation: evidence from California," Climatic Change, Springer, vol. 176(4), pages 1-20, April.
    11. Boris Quétard & Nicolas Spatola & Benjamin A Parris & Ludovic Ferrand & Maria Augustinova, 2023. "A mouse-tracking study of the composite nature of the Stroop effect at the level of response execution," PLOS ONE, Public Library of Science, vol. 18(1), pages 1-26, January.
    12. Amand Führer & Daniel Tiller & Patrick Brzoska & Marie Korn & Christine Gröger & Andreas Wienke, 2019. "Health-Related Disparities among Migrant Children at School Entry in Germany. How does the Definition of Migration Status Matter?," IJERPH, MDPI, vol. 17(1), pages 1-16, December.
    13. Abbas, Sadia & Adapa, Sujana & Sheridan, Alison & Azeem, Muhammad Masood, 2022. "Informal competition and firm level innovation in South Asia: The moderating role of innovation time off and R&D intensity," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    14. Pardeshi, Shefali & Loughrey, Jason & O'Connor, Declan, 2024. "Farm Investment and Adoption of Fixed Milk Price Contracts on Irish Dairy Farms," German Journal of Agricultural Economics, Humboldt-Universitaet zu Berlin, Department for Agricultural Economics, vol. 73(1), April.
    15. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    16. Andrea Saltelli & Arnald Puy, 2023. "What can mathematical modelling contribute to a sociology of quantification?," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-8, December.
    17. Škrinjarić Bruno, 2023. "Competence Proximity to Employers’ Requirements and Labour Market Success of Economics and Business Graduates," South East European Journal of Economics and Business, Sciendo, vol. 18(2), pages 140-161, December.
    18. Duffy, Conor M.C. & Bú, Emerson Do & Pereira, Cícero Roberto & Madeira, Filipa & Hagiwara, Nao, 2024. "Healthcare providers’ psychological investment in clinical recommendations: Investigating the role of implicit racial attitudes," Social Science & Medicine, Elsevier, vol. 362(C).
    19. Oliver Lotter & Tobias Lieb & Jochen Molsner & Viktor Breul, 2021. "Predictors for Clinical Outcomes Related to Upper Extremity Musculoskeletal Disorders in a Healthy Working Population," IJERPH, MDPI, vol. 18(17), pages 1-15, August.
    20. Daniela Bellani & Luis Ortiz-Gervasi, 2022. "Parental time preferences and educational choices: The role of children’s gender and of social origin," Rationality and Society, , vol. 34(1), pages 96-125, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:3002129. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.