IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v16y2025i1d10.1038_s41467-025-63825-0.html
   My bibliography  Save this article

Transfer learning via distributed brain recordings enables reliable speech decoding

Author

Listed:
  • Aditya Singh

    (University of Texas Health Science Center at Houston
    University of Texas Health Science Center at Houston)

  • Tessy Thomas

    (University of Texas Health Science Center at Houston
    University of Texas Health Science Center at Houston)

  • Jinlong Li

    (University of Texas Health Science Center at Houston
    University of Texas Health Science Center at Houston
    Rice University)

  • Greg Hickok

    (University of California)

  • Xaq Pitkow

    (Carnegie Mellon University)

  • Nitin Tandon

    (University of Texas Health Science Center at Houston
    University of Texas Health Science Center at Houston
    Texas Medical Center)

Abstract

Speech brain-computer interfaces (BCIs) combine neural recordings with large language models to achieve real-time intelligible speech. However, these decoders rely on dense, intact cortical coverage and are challenging to scale across individuals with heterogeneous brain organization. To derive scalable transfer learning strategies for neural speech decoding, we used minimally invasive stereo-electroencephalography recordings in a large cohort performing a demanding speech motor task. A sequence-to-sequence model enabled decoding of variable-length phonemic sequences prior to and during articulation. This enabled development of a cross-subject transfer learning framework to isolate shared latent manifolds while enabling individual model initialization. The group-derived decoder significantly outperformed models trained on individual data alone, enabling decoding robustness despite variable coverage and activation. These results highlight a pathway toward generalizable neural prostheses for speech and language disorders by leveraging large-scale intracranial datasets with distributed spatial sampling and shared task demands.

Suggested Citation

  • Aditya Singh & Tessy Thomas & Jinlong Li & Greg Hickok & Xaq Pitkow & Nitin Tandon, 2025. "Transfer learning via distributed brain recordings enables reliable speech decoding," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
  • Handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-63825-0
    DOI: 10.1038/s41467-025-63825-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-025-63825-0
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-025-63825-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Francis R. Willett & Erin M. Kunz & Chaofei Fan & Donald T. Avansino & Guy H. Wilson & Eun Young Choi & Foram Kamdar & Matthew F. Glasser & Leigh R. Hochberg & Shaul Druckmann & Krishna V. Shenoy & Ja, 2023. "A high-performance speech neuroprosthesis," Nature, Nature, vol. 620(7976), pages 1031-1036, August.
    2. Mostafa Safaie & Joanna C. Chang & Junchol Park & Lee E. Miller & Joshua T. Dudman & Matthew G. Perich & Juan A. Gallego, 2023. "Preserved neural dynamics across animals performing similar behaviour," Nature, Nature, vol. 623(7988), pages 765-771, November.
    3. Suseendrakumar Duraivel & Shervin Rahimpour & Chia-Han Chiang & Michael Trumpis & Charles Wang & Katrina Barth & Stephen C. Harward & Shivanand P. Lad & Allan H. Friedman & Derek G. Southwell & Saurab, 2023. "High-resolution neural recordings improve the accuracy of speech decoding," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    4. Arjun R. Khanna & William Muñoz & Young Joon Kim & Yoav Kfir & Angelique C. Paulk & Mohsen Jamali & Jing Cai & Martina L. Mustroph & Irene Caprara & Richard Hardstone & Mackenna Mejdell & Domokos Mesz, 2024. "Single-neuronal elements of speech production in humans," Nature, Nature, vol. 626(7999), pages 603-610, February.
    5. Kristofer E. Bouchard & Nima Mesgarani & Keith Johnson & Edward F. Chang, 2013. "Functional organization of human sensorimotor cortex for speech articulation," Nature, Nature, vol. 495(7441), pages 327-332, March.
    6. Kristofer E. Bouchard & Nima Mesgarani & Keith Johnson & Edward F. Chang, 2013. "Correction: Corrigendum: Functional organization of human sensorimotor cortex for speech articulation," Nature, Nature, vol. 498(7455), pages 526-526, June.
    7. Sean L. Metzger & Jessie R. Liu & David A. Moses & Maximilian E. Dougherty & Margaret P. Seaton & Kaylo T. Littlejohn & Josh Chartier & Gopala K. Anumanchipalli & Adelyn Tu-Chan & Karunesh Ganguly & E, 2022. "Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    8. Oscar Woolnough & Cristian Donos & Patrick S. Rollo & Kiefer J. Forseth & Yair Lakretz & Nathan E. Crone & Simon Fischer-Baum & Stanislas Dehaene & Nitin Tandon, 2021. "Spatiotemporal dynamics of orthographic and lexical processing in the ventral visual pathway," Nature Human Behaviour, Nature, vol. 5(3), pages 389-398, March.
    9. Gopala K. Anumanchipalli & Josh Chartier & Edward F. Chang, 2019. "Speech synthesis from neural decoding of spoken sentences," Nature, Nature, vol. 568(7753), pages 493-498, April.
    10. Sean L. Metzger & Kaylo T. Littlejohn & Alexander B. Silva & David A. Moses & Margaret P. Seaton & Ran Wang & Maximilian E. Dougherty & Jessie R. Liu & Peter Wu & Michael A. Berger & Inga Zhuravleva &, 2023. "A high-performance neuroprosthesis for speech decoding and avatar control," Nature, Nature, vol. 620(7976), pages 1037-1046, August.
    11. Jason K. Hsieh & Prashanth R. Prakash & Robert D. Flint & Zachary Fitzgerald & Emily Mugler & Yujing Wang & Nathan E. Crone & Jessica W. Templer & Joshua M. Rosenow & Matthew C. Tate & Richard Betzel , 2024. "Cortical sites critical to language function act as connectors between language subnetworks," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Junfeng Lu & Yuanning Li & Zehao Zhao & Yan Liu & Yanming Zhu & Ying Mao & Jinsong Wu & Edward F. Chang, 2023. "Neural control of lexical tone production in human laryngeal motor cortex," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    2. Sarah K. Wandelt & David A. Bjånes & Kelsie Pejsa & Brian Lee & Charles Liu & Richard A. Andersen, 2024. "Representation of internal speech by single neurons in human supramarginal gyrus," Nature Human Behaviour, Nature, vol. 8(6), pages 1136-1149, June.
    3. Suseendrakumar Duraivel & Shervin Rahimpour & Chia-Han Chiang & Michael Trumpis & Charles Wang & Katrina Barth & Stephen C. Harward & Shivanand P. Lad & Allan H. Friedman & Derek G. Southwell & Saurab, 2023. "High-resolution neural recordings improve the accuracy of speech decoding," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    4. Sean L. Metzger & Jessie R. Liu & David A. Moses & Maximilian E. Dougherty & Margaret P. Seaton & Kaylo T. Littlejohn & Josh Chartier & Gopala K. Anumanchipalli & Adelyn Tu-Chan & Karunesh Ganguly & E, 2022. "Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    5. Stanisz, Tomasz & Drożdż, Stanisław & Kwapień, Jarosław, 2023. "Universal versus system-specific features of punctuation usage patterns in major Western languages," Chaos, Solitons & Fractals, Elsevier, vol. 168(C).
    6. Anna Mai & Stephanie Riès & Sharona Ben-Haim & Jerry J. Shih & Timothy Q. Gentner, 2024. "Acoustic and language-specific sources for phonemic abstraction from speech," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    7. Dimitris Boufidis & Raghav Garg & Eugenia Angelopoulos & D. Kacy Cullen & Flavia Vitale, 2025. "Bio-inspired electronics: Soft, biohybrid, and “living” neural interfaces," Nature Communications, Nature, vol. 16(1), pages 1-17, December.
    8. Lingyun Zhao & Xiaoqin Wang, 2023. "Frontal cortex activity during the production of diverse social communication calls in marmoset monkeys," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    9. Matteo Vissani & Alan Bush & Witold J. Lipski & Latané Bullock & Petra Fischer & Clemens Neudorfer & Lori L. Holt & Julie A. Fiez & Robert S. Turner & R. Mark Richardson, 2025. "Spike-phase coupling of subthalamic neurons to posterior perisylvian cortex predicts speech sound accuracy," Nature Communications, Nature, vol. 16(1), pages 1-20, December.
    10. Jizhi Liang & Xiner Wang & Zhaohan Chen & Xiaoling Wei & Liuyang Sun & Keyin Liu & Zhifeng Shi & Tiger H. Tao & Zhitao Zhou, 2025. "Silk-enabled conformal intraventricular interfaces for minimally invasive neural recordings," Nature Communications, Nature, vol. 16(1), pages 1-14, December.
    11. Ah-Hyoung Lee & Jihun Lee & Vincent Leung & Lawrence Larson & Arto Nurmikko, 2024. "Patterned electrical brain stimulation by a wireless network of implantable microdevices," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    12. Bao Le & Margot C. Bjoring & C. Daniel Meliza, 2025. "The zebra finch auditory cortex reconstructs occluded syllables in conspecific song," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    13. Sijia Xu & Jie-Xiang Yu & Hongshuang Guo & Shu Tian & You Long & Jing Yang & Lei Zhang, 2023. "Force-induced ion generation in zwitterionic hydrogels for a sensitive silent-speech sensor," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    14. Tao Liu & Mingyang Zhang & Zhihao Li & Hanjie Dou & Wangyang Zhang & Jiaqian Yang & Pengfan Wu & Dongxiao Li & Xiaojing Mu, 2025. "Machine learning-assisted wearable sensing systems for speech recognition and interaction," Nature Communications, Nature, vol. 16(1), pages 1-13, December.
    15. Sai Deepesh Pokala & Marie Bernert & Takuya Nanami & Takashi Kohno & Timothée Lévi & Blaise Yvert, 2025. "A frugal Spiking Neural Network for unsupervised multivariate temporal pattern classification and multichannel spike sorting," Nature Communications, Nature, vol. 16(1), pages 1-16, December.
    16. You Wang & Ming Zhang & Ruifen Hu & Guang Li & Nan Li, 2020. "Silent Speech Recognition for BCI - A Review," Biomedical Journal of Scientific & Technical Research, Biomedical Research Network+, LLC, vol. 27(2), pages 20625-20627, April.
    17. Michael Kai Petersen, 2015. "Latent Semantics of Action Verbs Reflect Phonetic Parameters of Intensity and Emotional Content," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    18. Shanqing Cai & Subhashini Venugopalan & Katie Seaver & Xiang Xiao & Katrin Tomanek & Sri Jalasutram & Meredith Ringel Morris & Shaun Kane & Ajit Narayanan & Robert L. MacDonald & Emily Kornman & Danie, 2024. "Using large language models to accelerate communication for eye gaze typing users with ALS," Nature Communications, Nature, vol. 15(1), pages 1-18, December.
    19. Benjamin R Cowley & Matthew A Smith & Adam Kohn & Byron M Yu, 2016. "Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex," PLOS Computational Biology, Public Library of Science, vol. 12(12), pages 1-31, December.
    20. Xiao-yu Sun & Bin Ye, 2023. "The functional differentiation of brain–computer interfaces (BCIs) and its ethical implications," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-9, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-63825-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.