IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1012474.html
   My bibliography  Save this article

Roles and interplay of reinforcement-based and error-based processes during reaching and gait in neurotypical adults and individuals with Parkinson’s disease

Author

Listed:
  • Adam M Roth
  • John H Buggeln
  • Joanna E Hoh
  • Jonathan M Wood
  • Seth R Sullivan
  • Truc T Ngo
  • Jan A Calalo
  • Rakshith Lokesh
  • Susanne M Morton
  • Stephen Grill
  • John J Jeka
  • Michael J Carter
  • Joshua G A Cashaback

Abstract

From a game of darts to neurorehabilitation, the ability to explore and fine tune our movements is critical for success. Past work has shown that exploratory motor behaviour in response to reinforcement (reward) feedback is closely linked with the basal ganglia, while movement corrections in response to error feedback is commonly attributed to the cerebellum. While our past work has shown these processes are dissociable during adaptation, it is unknown how they uniquely impact exploratory behaviour. Moreover, converging neuroanatomical evidence shows direct and indirect connections between the basal ganglia and cerebellum, suggesting that there is an interaction between reinforcement-based and error-based neural processes. Here we examine the unique roles and interaction between reinforcement-based and error-based processes on sensorimotor exploration in a neurotypical population. We also recruited individuals with Parkinson’s disease to gain mechanistic insight into the role of the basal ganglia and associated reinforcement pathways in sensorimotor exploration. Across three reaching experiments, participants were given either reinforcement feedback, error feedback, or simultaneously both reinforcement & error feedback during a sensorimotor task that encouraged exploration. Our reaching results, a re-analysis of a previous gait experiment, and our model suggests that in isolation, reinforcement-based and error-based processes respectively boost and suppress exploration. When acting in concert, we found that reinforcement-based and error-based processes interact by mutually opposing one another. Finally, we found that those with Parkinson’s disease had decreased exploration when receiving reinforcement feedback, supporting the notion that compromised reinforcement-based processes reduces the ability to explore new motor actions. Understanding the unique and interacting roles of reinforcement-based and error-based processes may help to inform neurorehabilitation paradigms where it is important to discover new and successful motor actions.Author summary: Reinforcement-based and error-based processes play a pivotal role in regulating our movements. Converging neuroanatomical evidence show interconnected reinforcement-based and error-based neural circuits. Yet is unclear how reinforcement-based and error-based processes interact to influence sensorimotor behavior. In our past work we showed that reinforcement-based and error-based processes are dissociable. Building on this work, here we show that these process can also interact to influence trial-by-trial sensorimotor behaviour.

Suggested Citation

  • Adam M Roth & John H Buggeln & Joanna E Hoh & Jonathan M Wood & Seth R Sullivan & Truc T Ngo & Jan A Calalo & Rakshith Lokesh & Susanne M Morton & Stephen Grill & John J Jeka & Michael J Carter & Josh, 2024. "Roles and interplay of reinforcement-based and error-based processes during reaching and gait in neurotypical adults and individuals with Parkinson’s disease," PLOS Computational Biology, Public Library of Science, vol. 20(10), pages 1-32, October.
  • Handle: RePEc:plo:pcbi00:1012474
    DOI: 10.1371/journal.pcbi.1012474
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012474
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1012474&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1012474?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jonathan B Dingwell & Joby John & Joseph P Cusumano, 2010. "Do Humans Optimally Exploit Redundancy to Control Step Variability in Walking?," PLOS Computational Biology, Public Library of Science, vol. 6(7), pages 1-15, July.
    2. Kurt A. Thoroughman & Reza Shadmehr, 2000. "Learning of action through adaptive combination of motor primitives," Nature, Nature, vol. 407(6805), pages 742-747, October.
    3. Dagmar Sternad & Se-Woong Park & Hermann Müller & Neville Hogan, 2010. "Coordinate Dependence of Variability Analysis," PLOS Computational Biology, Public Library of Science, vol. 6(4), pages 1-16, April.
    4. Mathias Pessiglione & Ben Seymour & Guillaume Flandin & Raymond J. Dolan & Chris D. Frith, 2006. "Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans," Nature, Nature, vol. 442(7106), pages 1042-1045, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Manuel Glauco Carbone & Icro Maremmani, 2024. "Chronic Cocaine Use and Parkinson’s Disease: An Interpretative Model," IJERPH, MDPI, vol. 21(8), pages 1-23, August.
    2. repec:plo:pone00:0037900 is not listed on IDEAS
    3. Hanan Shteingart & Tal Neiman & Yonatan Loewenstein, 2012. "The Role of First Impression in Operant Learning," Discussion Paper Series dp626, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    4. Kuhnen, Camelia M. & Miu, Andrei C., 2017. "Socioeconomic status and learning from financial information," Journal of Financial Economics, Elsevier, vol. 124(2), pages 349-372.
    5. Reimann, Martin & Bechara, Antoine, 2010. "The somatic marker framework as a neurological theory of decision-making: Review, conceptual comparisons, and future neuroeconomics research," Journal of Economic Psychology, Elsevier, vol. 31(5), pages 767-776, October.
    6. George I. Christopoulos & Xiao-Xiao Liu & Ying-yi Hong, 2017. "Toward an Understanding of Dynamic Moral Decision Making: Model-Free and Model-Based Learning," Journal of Business Ethics, Springer, vol. 144(4), pages 699-715, September.
    7. Steven H Collins & Arthur D Kuo, 2013. "Two Independent Contributions to Step Variability during Over-Ground Human Walking," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-1, August.
    8. Maël Lebreton & Karin Bacily & Stefano Palminteri & Jan B Engelmann, 2019. "Contextual influence on confidence judgments in human reinforcement learning," PLOS Computational Biology, Public Library of Science, vol. 15(4), pages 1-27, April.
    9. repec:plo:pbio00:0040179 is not listed on IDEAS
    10. Barbara Feulner & Matthew G. Perich & Raeed H. Chowdhury & Lee E. Miller & Juan A. Gallego & Claudia Clopath, 2022. "Small, correlated changes in synaptic connectivity may facilitate rapid motor learning," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    11. Jonathan B Dingwell & Joseph P Cusumano, 2019. "Humans use multi-objective control to regulate lateral foot placement when walking," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-28, March.
    12. Paolo Tommasino & Antonella Maselli & Domenico Campolo & Francesco Lacquaniti & Andrea d’Avella, 2021. "A Hessian-based decomposition characterizes how performance in complex motor skills depends on individual strategy and variability," PLOS ONE, Public Library of Science, vol. 16(6), pages 1-32, June.
    13. Joshua G A Cashaback & Christopher K Lao & Dimitrios J Palidis & Susan K Coltman & Heather R McGregor & Paul L Gribble, 2019. "The gradient of the reinforcement landscape influences sensorimotor learning," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-27, March.
    14. Yael Mandelblat-Cerf & Itai Novick & Eilon Vaadia, 2011. "Expressions of Multiple Neuronal Dynamics during Sensorimotor Learning in the Motor Cortex of Behaving Monkeys," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-14, July.
    15. Joshua G A Cashaback & Heather R McGregor & Ayman Mohatarem & Paul L Gribble, 2017. "Dissociating error-based and reinforcement-based loss functions during sensorimotor learning," PLOS Computational Biology, Public Library of Science, vol. 13(7), pages 1-28, July.
    16. repec:plo:pone00:0086580 is not listed on IDEAS
    17. Kuhnen, Camelia M. & Knutson, Brian, 2011. "The Influence of Affect on Beliefs, Preferences, and Financial Decisions," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 46(3), pages 605-626, June.
    18. Bruno B Averbeck, 2015. "Theory of Choice in Bandit, Information Sampling and Foraging Tasks," PLOS Computational Biology, Public Library of Science, vol. 11(3), pages 1-28, March.
    19. Karima Chakroun & Antonius Wiehler & Ben Wagner & David Mathar & Florian Ganzer & Thilo Eimeren & Tobias Sommer & Jan Peters, 2023. "Dopamine regulates decision thresholds in human reinforcement learning in males," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    20. Sepulveda, Pradyumna & Aitsahalia, Ines & Kumar, Krishan & Atkin, Tobias & Iigaya, Kiyohito, 2024. "Addressing Altered Anticipation as a Transdiagnostic Target through Computational Psychiatry," OSF Preprints dtm3r, Center for Open Science.
    21. Stefano Palminteri & Emma J Kilford & Giorgio Coricelli & Sarah-Jayne Blakemore, 2016. "The Computational Development of Reinforcement Learning during Adolescence," PLOS Computational Biology, Public Library of Science, vol. 12(6), pages 1-25, June.
    22. Daniel Blustein & Ahmed Shehata & Kevin Englehart & Jonathon Sensinger, 2018. "Conventional analysis of trial-by-trial adaptation is biased: Empirical and theoretical support using a Bayesian estimator," PLOS Computational Biology, Public Library of Science, vol. 14(12), pages 1-15, December.
    23. Julius Verrel & Didier Pradon & Nicolas Vuillerme, 2012. "Persistence of Motor-Equivalent Postural Fluctuations during Bipedal Quiet Standing," PLOS ONE, Public Library of Science, vol. 7(10), pages 1-8, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1012474. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.