Predicting change: Approximate inference under explicit representation of temporal structure in changing environments

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants’ beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants’ behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior.Author summary: Although time perception and timed behavior are essential for our everyday experience, it is still unclear how the human brain represents the underlying temporal regularities of our dynamic environment. These regularities and their representations in the brain are important to generate well-timed behavior. When deciding on the sequence of actions to complete most of our everyday tasks like cooking, driving, or even brushing our teeth, it is essential to represent and keep track of the durations of different parts of the tasks. Here we introduce a behavioral model of decision making in environments in which a change is at least partially predictable by the time it took since the last change. We show that human participants are using such predictions in the so-called reversal learning task, which simulates abrupt but not immediately obvious changes of the environment. We find that some but not all participants harness previously experienced regularities in these changes to anticipate when the next change is going to happen. We expect that a wide range of similar questions of how humans and other animals use temporal expectations to make their decisions in a dynamic environment can be addressed using the new modelling approach.