Ambiguity drives higher-order Pavlovian learning

In the natural world, stimulus-outcome associations are often ambiguous, and most associations are highly complex and situation-dependent. Learning to disambiguate these complex associations to identify which specific outcomes will occur in which situations is critical for survival. Pavlovian occasion setters are stimuli that determine whether other stimuli will result in a specific outcome. Occasion setting is a well-established phenomenon, but very little investigation has been conducted on how occasion setters are disambiguated when they themselves are ambiguous (i.e., when they do not consistently signal whether another stimulus will be reinforced). In two preregistered studies, we investigated the role of higher-order Pavlovian occasion setting in humans. We developed and tested the first computational model predicting direct associative learning, traditional occasion setting (i.e., 1st-order occasion setting), and 2nd-order occasion setting. This model operationalizes stimulus ambiguity as a mechanism to engage in higher-order Pavlovian learning. Both behavioral and computational modeling results suggest that 2nd-order occasion setting was learned, as evidenced by lack and presence of transfer of occasion setting properties when expected and the superior fit of our 2nd-order occasion setting model compared to the 1st-order occasion setting or direct associations models. These results provide a controlled investigation into highly complex associative learning and may ultimately lead to improvements in the treatment of Pavlovian-based mental health disorders (e.g., anxiety disorders, substance use).Author summary: In everyday life, we learn to associate various situations with various outcomes. For example, perhaps a specific person usually receives praise (outcome) when giving a public speech (situation), but if they give a speech after a particularly charismatic speaker (situational factor), their speech might receive less praise than usual. In our report, we conducted two experiments investigating how people learn about highly complex associations. By highly complex, we mean that whether stimulus 1 (S1) leads to outcome 1 (O1) depends on whether stimulus 2 (S2) was present or absent, and S2’s effect on the S1/O1 association may further depend on whether stimulus 3 (S3) was present. Previous work has shown that S2 will modulate S1’s association with O1 using two hierarchies of stimulus value, where S2 is a higher-order stimulus that affects the lower-order S1/O1 association. We hypothesized a third hierarchical value exists if S3 affects how S2 modulates the S1/OS1 association. We created formulas predicting three hierarchical levels of learning, and we tested our three-level hypothesis and formulas in two experiments, finding strong support for both. Through the experiments, we show that humans have the ability to learn highly complex associations, which suggests we may do so in everyday life. Our formulas provide insight into the computations our brains may engage in while learning. This has relevance for increasing our understanding of associative learning (which has been largely limited to a two-level hierarchy to date), as well as mental health disorders related to associative learning (e.g., anxiety, substance use).