Mechanisms of sensorimotor adaptation in a hierarchical state feedback control model of speech

Upon perceiving sensory errors during movements, the human sensorimotor system updates future movements to compensate for the errors, a phenomenon called sensorimotor adaptation. One component of this adaptation is thought to be driven by sensory prediction errors–discrepancies between predicted and actual sensory feedback. However, the mechanisms by which prediction errors drive adaptation remain unclear. Here, auditory prediction error-based mechanisms involved in speech auditory-motor adaptation were examined via the feedback aware control of tasks in speech (FACTS) model. Consistent with theoretical perspectives in both non-speech and speech motor control, the hierarchical architecture of FACTS relies on both the higher-level task (vocal tract constrictions) as well as lower-level articulatory state representations. Importantly, FACTS also computes sensory prediction errors as a part of its state feedback control mechanism, a well-established framework in the field of motor control. We explored potential adaptation mechanisms and found that adaptive behavior was present only when prediction errors updated the articulatory-to-task state transformation. In contrast, designs in which prediction errors updated forward sensory prediction models alone did not generate adaptation. Thus, FACTS demonstrated that 1) prediction errors can drive adaptation through task-level updates, and 2) adaptation is likely driven by updates to task-level control rather than (only) to forward predictive models. Additionally, simulating adaptation with FACTS generated a number of important hypotheses regarding previously reported phenomena such as identifying the source(s) of incomplete adaptation and driving factor(s) for changes in the second formant frequency during adaptation to the first formant perturbation. The proposed model design paves the way for a hierarchical state feedback control framework to be examined in the context of sensorimotor adaptation in both speech and non-speech effector systems.Author summary: When we move, our brain predicts the sensory feedback that would result from the movement, and can quickly adjust future movements based on any sensory prediction errors—differences between the predictions and actual sensory feedback. This learning process, sensorimotor adaptation, has been extensively studied in many movements (e.g., walking, reaching, speaking), but its underlying mechanisms remain largely unclear. Here, we examined mechanisms driving speech adaptation in response to altered auditory feedback using the FACTS model, a hierarchical state feedback control model of speech in which a high-level controller achieves speech goals (e.g., constrictions of the vocal tract) by directing a low-level controller that moves the speech articulators (e.g., positions of the jaw and the tongue). We demonstrated that prediction errors can drive adaptation through changes in high-level control, but not solely through changes in predictions of movement outcomes or low-level control. In addition to replicating multiple key features of sensorimotor adaptation in speech, our simulations also generated potential new explanations for phenomena that are currently poorly understood. Importantly, given that our model design is closely aligned with widely accepted motor control frameworks outside of speech, these results have the potential to be broadly applicable to non-speech motor systems as well.