IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-44248-1.html
   My bibliography  Save this article

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

Author

Listed:
  • Colin W. Hoy

    (University of California, San Francisco
    University of California, Berkeley)

  • David R. Quiroga-Martinez

    (University of California, Berkeley
    Aarhus University & The Royal Academy of Music)

  • Eduardo Sandoval

    (University of California, Berkeley)

  • David King-Stephens

    (California Pacific Medical Center
    Yale School of Medicine)

  • Kenneth D. Laxer

    (California Pacific Medical Center)

  • Peter Weber

    (California Pacific Medical Center)

  • Jack J. Lin

    (University of California, Davis
    University of California, Davis)

  • Robert T. Knight

    (University of California, Berkeley
    University of California, Berkeley)

Abstract

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Suggested Citation

  • Colin W. Hoy & David R. Quiroga-Martinez & Eduardo Sandoval & David King-Stephens & Kenneth D. Laxer & Peter Weber & Jack J. Lin & Robert T. Knight, 2023. "Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-44248-1
    DOI: 10.1038/s41467-023-44248-1
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-44248-1
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-44248-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ian R. Kleckner & Jiahe Zhang & Alexandra Touroutoglou & Lorena Chanes & Chenjie Xia & W. Kyle Simmons & Karen S. Quigley & Bradford C. Dickerson & Lisa Feldman Barrett, 2017. "Evidence for a large-scale brain system supporting allostasis and interoception in humans," Nature Human Behaviour, Nature, vol. 1(5), pages 1-14, May.
    2. Marco K. Wittmann & Elsa Fouragnan & Davide Folloni & Miriam C. Klein-Flügge & Bolton K. H. Chau & Mehdi Khamassi & Matthew F. S. Rushworth, 2020. "Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys," Nature Communications, Nature, vol. 11(1), pages 1-17, December.
    3. Weidong Cai & Srikanth Ryali & Ramkrishna Pasumarthy & Viswanath Talasila & Vinod Menon, 2021. "Dynamic causal brain circuits during working memory and their functional controllability," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    4. You-Ping Yang & Xinjian Li & Veit Stuphorn, 2022. "Primate anterior insular cortex represents economic decision variables proposed by prospect theory," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    5. Jiefeng Jiang & Jeffrey Beck & Katherine Heller & Tobias Egner, 2015. "An insula-frontostriatal network mediates flexible cognitive control by adaptively predicting changing control demands," Nature Communications, Nature, vol. 6(1), pages 1-11, November.
    6. Romy Frömer & Carolyn K. Dean Wolf & Amitai Shenhav, 2019. "Goal congruency dominates reward value in accounting for behavioral and neural correlates of value-based decision-making," Nature Communications, Nature, vol. 10(1), pages 1-11, December.
    7. Ben Engelhard & Joel Finkelstein & Julia Cox & Weston Fleming & Hee Jae Jang & Sharon Ornelas & Sue Ann Koay & Stephan Y. Thiberge & Nathaniel D. Daw & David W. Tank & Ilana B. Witten, 2019. "Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons," Nature, Nature, vol. 570(7762), pages 509-513, June.
    8. Erin L. Rich & Joni D. Wallis, 2017. "Spatiotemporal dynamics of information encoding revealed in orbitofrontal high-gamma," Nature Communications, Nature, vol. 8(1), pages 1-13, December.
    9. Masayuki Matsumoto & Okihide Hikosaka, 2009. "Two types of dopamine neuron distinctly convey positive and negative motivational signals," Nature, Nature, vol. 459(7248), pages 837-841, June.
    10. Will Dabney & Zeb Kurth-Nelson & Naoshige Uchida & Clara Kwon Starkweather & Demis Hassabis & Rémi Munos & Matthew Botvinick, 2020. "A distributional code for value in dopamine-based reinforcement learning," Nature, Nature, vol. 577(7792), pages 671-675, January.
    11. Ilya E. Monosov, 2017. "Anterior cingulate is a source of valence-specific information about value and uncertainty," Nature Communications, Nature, vol. 8(1), pages 1-12, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vincent Man & Jeffrey Cockburn & Oliver Flouty & Phillip E. Gander & Masahiro Sawada & Christopher K. Kovach & Hiroto Kawasaki & Hiroyuki Oya & Matthew A. Howard III & John P. O’Doherty, 2024. "Temporally organized representations of reward and risk in the human brain," Nature Communications, Nature, vol. 15(1), pages 1-16, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wan-Yu Shih & Hsiang-Yu Yu & Cheng-Chia Lee & Chien-Chen Chou & Chien Chen & Paul W. Glimcher & Shih-Wei Wu, 2023. "Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex," Nature Communications, Nature, vol. 14(1), pages 1-24, December.
    2. Torben Ott & Anna Marlina Stein & Andreas Nieder, 2023. "Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    3. Leo Chi U Seak & Simone Ferrari-Toniolo & Ritesh Jain & Kirby Nielsen & Wolfram Schultz, 2023. "Systematic comparison of risky choices in humans and monkeys," Working Papers 202316, University of Liverpool, Department of Economics.
    4. Jan Weber & Anne-Kristin Solbakk & Alejandro O. Blenkmann & Anais Llorens & Ingrid Funderud & Sabine Leske & Pål Gunnar Larsson & Jugoslav Ivanovic & Robert T. Knight & Tor Endestad & Randolph F. Helf, 2024. "Ramping dynamics and theta oscillations reflect dissociable signatures during rule-guided human behavior," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    5. Frederick Callaway & Antonio Rangel & Thomas L Griffiths, 2021. "Fixation patterns in simple choice reflect optimal information sampling," PLOS Computational Biology, Public Library of Science, vol. 17(3), pages 1-29, March.
    6. Hiroyuki Kawai & Youcef Bouchekioua & Naoya Nishitani & Kazuhei Niitani & Shoma Izumi & Hinako Morishita & Chihiro Andoh & Yuma Nagai & Masashi Koda & Masako Hagiwara & Koji Toda & Hisashi Shirakawa &, 2022. "Median raphe serotonergic neurons projecting to the interpeduncular nucleus control preference and aversion," Nature Communications, Nature, vol. 13(1), pages 1-22, December.
    7. Dimitrije Marković & Andrea M F Reiter & Stefan J Kiebel, 2019. "Predicting change: Approximate inference under explicit representation of temporal structure in changing environments," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-31, January.
    8. Hong Yu & Xinkuan Xiang & Zongming Chen & Xu Wang & Jiaqi Dai & Xinxin Wang & Pengcheng Huang & Zheng-dong Zhao & Wei L. Shen & Haohong Li, 2021. "Periaqueductal gray neurons encode the sequential motor program in hunting behavior of mice," Nature Communications, Nature, vol. 12(1), pages 1-15, December.
    9. John N. J. Reynolds & Riccardo Avvisati & Paul D. Dodson & Simon D. Fisher & Manfred J. Oswald & Jeffery R. Wickens & Yan-Feng Zhang, 2022. "Coincidence of cholinergic pauses, dopaminergic activation and depolarisation of spiny projection neurons drives synaptic plasticity in the striatum," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    10. Yosuke Yawata & Yu Shikano & Jun Ogasawara & Kenichi Makino & Tetsuhiko Kashima & Keiko Ihara & Airi Yoshimoto & Shota Morikawa & Sho Yagishita & Kenji F. Tanaka & Yuji Ikegaya, 2023. "Mesolimbic dopamine release precedes actively sought aversive stimuli in mice," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    11. Kazutaka Maeda & Ken-ichi Inoue & Masahiko Takada & Okihide Hikosaka, 2023. "Environmental context-dependent activation of dopamine neurons via putative amygdala-nigra pathway in macaques," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    12. Kyuhyun Choi & Eugenio Piasini & Edgar Díaz-Hernández & Luigim Vargas Cifuentes & Nathan T. Henderson & Elizabeth N. Holly & Manivannan Subramaniyan & Charles R. Gerfen & Marc V. Fuccillo, 2023. "Distributed processing for value-based choice by prelimbic circuits targeting anterior-posterior dorsal striatal subregions in male mice," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    13. Elizabeth L. Johnson & Jack J. Lin & David King-Stephens & Peter B. Weber & Kenneth D. Laxer & Ignacio Saez & Fady Girgis & Mark D’Esposito & Robert T. Knight & David Badre, 2023. "A rapid theta network mechanism for flexible information encoding," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    14. Vinod Menon & Domenic Cerri & Byeongwook Lee & Rui Yuan & Sung-Ho Lee & Yen-Yu Ian Shih, 2023. "Optogenetic stimulation of anterior insular cortex neurons in male rats reveals causal mechanisms underlying suppression of the default mode network by the salience network," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    15. Xin-Yue Wang & Wen-Bin Jia & Xiang Xu & Rui Chen & Liang-Biao Wang & Xiao-Jing Su & Peng-Fei Xu & Xiao-Qing Liu & Jie Wen & Xiao-Yuan Song & Yuan-Yuan Liu & Zhi Zhang & Xin-Feng Liu & Yan Zhang, 2023. "A glutamatergic DRN–VTA pathway modulates neuropathic pain and comorbid anhedonia-like behavior in mice," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    16. Ruth Pauli & Inti A. Brazil & Gregor Kohls & Miriam C. Klein-Flügge & Jack C. Rogers & Dimitris Dikeos & Roberta Dochnal & Graeme Fairchild & Aranzazu Fernández-Rivas & Beate Herpertz-Dahlmann & Amaia, 2023. "Action initiation and punishment learning differ from childhood to adolescence while reward learning remains stable," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    17. Minkyu Shin & Jin Kim & Minkyung Kim, 2020. "Measuring Human Adaptation to AI in Decision Making: Application to Evaluate Changes after AlphaGo," Papers 2012.15035, arXiv.org, revised Jan 2021.
    18. Ruben Bosch & Britt Lambregts & Jessica Määttä & Lieke Hofmans & Danae Papadopetraki & Andrew Westbrook & Robbert-Jan Verkes & Jan Booij & Roshan Cools, 2022. "Striatal dopamine dissociates methylphenidate effects on value-based versus surprise-based reversal learning," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    19. A. Calapai & J. Cabrera-Moreno & T. Moser & M. Jeschke, 2022. "Flexible auditory training, psychophysics, and enrichment of common marmosets with an automated, touchscreen-based system," Nature Communications, Nature, vol. 13(1), pages 1-16, December.
    20. Kazuo Sano, 2022. "New Concept for the Value Function of Prospect Theory," Papers 2211.00131, arXiv.org, revised Jun 2024.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-44248-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.