IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v16y2025i1d10.1038_s41467-025-58606-8.html
   My bibliography  Save this article

Data splitting to avoid information leakage with DataSAIL

Author

Listed:
  • Roman Joeres

    (Helmholtz Centre for Infection Research (HZI)
    Saarland University
    University of Gothenburg
    University of Gothenburg)

  • David B. Blumenthal

    (Friedrich-Alexander-Universität Erlangen-Nürnberg)

  • Olga V. Kalinina

    (Helmholtz Centre for Infection Research (HZI)
    Saarland University
    Saarland University)

Abstract

Information leakage is an increasingly important topic in machine learning research for biomedical applications. When information leakage happens during a model’s training, it risks memorizing the training data instead of learning generalizable properties. This can lead to inflated performance metrics that do not reflect the actual performance at inference time. We present DataSAIL, a versatile Python package to facilitate leakage-reduced data splitting to enable realistic evaluation of machine learning models for biological data that are intended to be applied in out-of-distribution scenarios. DataSAIL is based on formulating the problem to find leakage-reduced data splits as a combinatorial optimization problem. We prove that this problem is NP-hard and provide a scalable heuristic based on clustering and integer linear programming. Finally, we empirically demonstrate DataSAIL’s impact on evaluating biomedical machine learning models.

Suggested Citation

  • Roman Joeres & David B. Blumenthal & Olga V. Kalinina, 2025. "Data splitting to avoid information leakage with DataSAIL," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
  • Handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-58606-8
    DOI: 10.1038/s41467-025-58606-8
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-025-58606-8
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-025-58606-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jianyuan Deng & Zhibo Yang & Hehe Wang & Iwao Ojima & Dimitris Samaras & Fusheng Wang, 2023. "A systematic study of key elements underlying molecular property prediction," Nature Communications, Nature, vol. 14(1), pages 1-20, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yue Wan & Jialu Wu & Tingjun Hou & Chang-Yu Hsieh & Xiaowei Jia, 2025. "Multi-channel learning for integrating structural hierarchies into context-dependent molecular representation," Nature Communications, Nature, vol. 16(1), pages 1-13, December.
    2. Ana Laura Dias & Latimah Bustillo & Tiago Rodrigues, 2023. "Limitations of representation learning in small molecule property prediction," Nature Communications, Nature, vol. 14(1), pages 1-2, December.
    3. Fleming Kretschmer & Jan Seipp & Marcus Ludwig & Gunnar W. Klau & Sebastian Böcker, 2025. "Coverage bias in small molecule machine learning," Nature Communications, Nature, vol. 16(1), pages 1-19, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-58606-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.