IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-42555-1.html
   My bibliography  Save this article

High-resolution neural recordings improve the accuracy of speech decoding

Author

Listed:
  • Suseendrakumar Duraivel

    (Duke University)

  • Shervin Rahimpour

    (Duke School of Medicine
    University of Utah)

  • Chia-Han Chiang

    (Duke University)

  • Michael Trumpis

    (Duke University)

  • Charles Wang

    (Duke University)

  • Katrina Barth

    (Duke University)

  • Stephen C. Harward

    (Duke School of Medicine
    Duke School of Medicine)

  • Shivanand P. Lad

    (Duke School of Medicine)

  • Allan H. Friedman

    (Duke School of Medicine)

  • Derek G. Southwell

    (Duke University
    Duke School of Medicine
    Duke School of Medicine
    Duke School of Medicine)

  • Saurabh R. Sinha

    (University of Pennsylvania)

  • Jonathan Viventi

    (Duke University
    Duke School of Medicine
    Duke School of Medicine
    Duke School of Medicine)

  • Gregory B. Cogan

    (Duke University
    Duke School of Medicine
    Duke School of Medicine
    Duke School of Medicine)

Abstract

Patients suffering from debilitating neurodegenerative diseases often lose the ability to communicate, detrimentally affecting their quality of life. One solution to restore communication is to decode signals directly from the brain to enable neural speech prostheses. However, decoding has been limited by coarse neural recordings which inadequately capture the rich spatio-temporal structure of human brain signals. To resolve this limitation, we performed high-resolution, micro-electrocorticographic (µECoG) neural recordings during intra-operative speech production. We obtained neural signals with 57× higher spatial resolution and 48% higher signal-to-noise ratio compared to macro-ECoG and SEEG. This increased signal quality improved decoding by 35% compared to standard intracranial signals. Accurate decoding was dependent on the high-spatial resolution of the neural interface. Non-linear decoding models designed to utilize enhanced spatio-temporal neural information produced better results than linear techniques. We show that high-density µECoG can enable high-quality speech decoding for future neural speech prostheses.

Suggested Citation

  • Suseendrakumar Duraivel & Shervin Rahimpour & Chia-Han Chiang & Michael Trumpis & Charles Wang & Katrina Barth & Stephen C. Harward & Shivanand P. Lad & Allan H. Friedman & Derek G. Southwell & Saurab, 2023. "High-resolution neural recordings improve the accuracy of speech decoding," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42555-1
    DOI: 10.1038/s41467-023-42555-1
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-42555-1
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-42555-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kristofer E. Bouchard & Nima Mesgarani & Keith Johnson & Edward F. Chang, 2013. "Functional organization of human sensorimotor cortex for speech articulation," Nature, Nature, vol. 495(7441), pages 327-332, March.
    2. David A. Moses & Matthew K. Leonard & Joseph G. Makin & Edward F. Chang, 2019. "Real-time decoding of question-and-answer speech dialogue using human cortical activity," Nature Communications, Nature, vol. 10(1), pages 1-14, December.
    3. Catherine A. Schevon & Shennan A. Weiss & Guy McKhann & Robert R. Goodman & Rafael Yuste & Ronald G. Emerson & Andrew J. Trevelyan, 2012. "Evidence of an inhibitory restraint of seizure activity in humans," Nature Communications, Nature, vol. 3(1), pages 1-11, January.
    4. Laura Gwilliams & Jean-Remi King & Alec Marantz & David Poeppel, 2022. "Neural dynamics of phoneme sequences reveal position-invariant code for content and order," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    5. Kristofer E. Bouchard & Nima Mesgarani & Keith Johnson & Edward F. Chang, 2013. "Correction: Corrigendum: Functional organization of human sensorimotor cortex for speech articulation," Nature, Nature, vol. 498(7455), pages 526-526, June.
    6. Gopala K. Anumanchipalli & Josh Chartier & Edward F. Chang, 2019. "Speech synthesis from neural decoding of spoken sentences," Nature, Nature, vol. 568(7753), pages 493-498, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sean L. Metzger & Jessie R. Liu & David A. Moses & Maximilian E. Dougherty & Margaret P. Seaton & Kaylo T. Littlejohn & Josh Chartier & Gopala K. Anumanchipalli & Adelyn Tu-Chan & Karunesh Ganguly & E, 2022. "Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    2. Junfeng Lu & Yuanning Li & Zehao Zhao & Yan Liu & Yanming Zhu & Ying Mao & Jinsong Wu & Edward F. Chang, 2023. "Neural control of lexical tone production in human laryngeal motor cortex," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    3. Stanisz, Tomasz & Drożdż, Stanisław & Kwapień, Jarosław, 2023. "Universal versus system-specific features of punctuation usage patterns in major Western languages," Chaos, Solitons & Fractals, Elsevier, vol. 168(C).
    4. Xiao-yu Sun & Bin Ye, 2023. "The functional differentiation of brain–computer interfaces (BCIs) and its ethical implications," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-9, December.
    5. Anna Mai & Stephanie Riès & Sharona Ben-Haim & Jerry J. Shih & Timothy Q. Gentner, 2024. "Acoustic and language-specific sources for phonemic abstraction from speech," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    6. Lingyun Zhao & Xiaoqin Wang, 2023. "Frontal cortex activity during the production of diverse social communication calls in marmoset monkeys," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    7. Sijia Xu & Jie-Xiang Yu & Hongshuang Guo & Shu Tian & You Long & Jing Yang & Lei Zhang, 2023. "Force-induced ion generation in zwitterionic hydrogels for a sensitive silent-speech sensor," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    8. Laura R González-Ramírez & Omar J Ahmed & Sydney S Cash & C Eugene Wayne & Mark A Kramer, 2015. "A Biologically Constrained, Mathematical Model of Cortical Wave Propagation Preceding Seizure Termination," PLOS Computational Biology, Public Library of Science, vol. 11(2), pages 1-34, February.
    9. You Wang & Ming Zhang & Ruifen Hu & Guang Li & Nan Li, 2020. "Silent Speech Recognition for BCI - A Review," Biomedical Journal of Scientific & Technical Research, Biomedical Research Network+, LLC, vol. 27(2), pages 20625-20627, April.
    10. Laurent Sheybani & Umesh Vivekananda & Roman Rodionov & Beate Diehl & Fahmida A. Chowdhury & Andrew W. McEvoy & Anna Miserocchi & James A. Bisby & Daniel Bush & Neil Burgess & Matthew C. Walker, 2023. "Wake slow waves in focal human epilepsy impact network activity and cognition," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    11. Michael Kai Petersen, 2015. "Latent Semantics of Action Verbs Reflect Phonetic Parameters of Intensity and Emotional Content," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    12. Benjamin R Cowley & Matthew A Smith & Adam Kohn & Byron M Yu, 2016. "Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex," PLOS Computational Biology, Public Library of Science, vol. 12(12), pages 1-31, December.
    13. Annika Hagemann & Jens Wilting & Bita Samimizad & Florian Mormann & Viola Priesemann, 2021. "Assessing criticality in pre-seizure single-neuron activity of human epileptic cortex," PLOS Computational Biology, Public Library of Science, vol. 17(3), pages 1-18, March.
    14. Taemin Kim & Yejee Shin & Kyowon Kang & Kiho Kim & Gwanho Kim & Yunsu Byeon & Hwayeon Kim & Yuyan Gao & Jeong Ryong Lee & Geonhui Son & Taeseong Kim & Yohan Jun & Jihyun Kim & Jinyoung Lee & Seyun Um , 2022. "Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    15. Ryan C Williamson & Benjamin R Cowley & Ashok Litwin-Kumar & Brent Doiron & Adam Kohn & Matthew A Smith & Byron M Yu, 2016. "Scaling Properties of Dimensionality Reduction for Neural Populations and Network Models," PLOS Computational Biology, Public Library of Science, vol. 12(12), pages 1-27, December.
    16. John-Sebastian Mueller & Fabio C. Tescarollo & Trong Huynh & Daniel A. Brenner & Daniel J. Valdivia & Kanyin Olagbegi & Sahana Sangappa & Spencer C. Chen & Hai Sun, 2023. "Ictogenesis proceeds through discrete phases in hippocampal CA1 seizures in mice," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    17. Yuki Bando & Michael Wenzel & Rafael Yuste, 2021. "Simultaneous two-photon imaging of action potentials and subthreshold inputs in vivo," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    18. Enrico Pracucci & Robert T. Graham & Laura Alberio & Gabriele Nardi & Olga Cozzolino & Vinoshene Pillai & Giacomo Pasquini & Luciano Saieva & Darren Walsh & Silvia Landi & Jinwei Zhang & Andrew J. Tre, 2023. "Daily rhythm in cortical chloride homeostasis underpins functional changes in visual cortex excitability," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    19. Prateek Jain & Alberto Garcia Garcia, 2022. "Quantum classical hybrid neural networks for continuous variable prediction," Papers 2212.04209, arXiv.org, revised Mar 2023.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42555-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.