Hierarchical recurrent temporal prediction as a model of the mammalian dorsal visual pathway

A major goal of neuroscience is to identify general principles that can explain the diverse structures and functions of the brain. The principle of temporal prediction provides one approach, arguing that the sensory brain is optimized to represent stimulus features that efficiently predict the immediate future input. Previous work has demonstrated that feedforward hierarchical temporal prediction models can capture the tuning properties of neurons along the visual pathway, and that recurrent temporal prediction models can explain local functional connectivity within primary visual cortex. However, the visual system is also characterized by extensive inter-areal feedback recurrency, which existing models lack. We aimed to better account for the dynamic features of neurons in the visual cortex by incorporating both local recurrency and inter-areal feedback connectivity into a hierarchical temporal prediction model. The resulting model captured tuning properties along the dorsal visual pathway, including pattern motion selectivity and surround suppression, and the contribution of inter-areal connectivity to these properties. Moreover, compared with several alternative normative models, the hierarchical recurrent temporal prediction model provided the closest fit to these tuning properties and was best able to explain neuronal responses to natural stimuli. Accordingly, temporal prediction accounts well for information processing along the visual pathway.Author summary: A goal of neuroscience is to identify general principles that explain how neural circuits represent information. Feedforward and locally recurrent models have shown that temporal prediction, the idea that sensory systems are optimized to predict immediate future input, can account for many aspects of the response properties and connectivity of sensory neurons. However, sensory pathways are also shaped by extensive feedback connections between brain regions, whose computational role remains poorly understood. Here, we address this gap by introducing a hierarchical recurrent temporal prediction model that explicitly incorporates inter-areal feedback, reflecting the bidirectional organization of mammalian cortex. Trained on natural movies, the model reproduces stimulus feature selectivity and local connectivity patterns observed in the dorsal visual cortical pathway. Critically, it also explains neural phenomena that depend on feedback, including enhanced memory, surround suppression, and the emergence of pattern-motion selectivity across the cortical processing hierarchy. Removing the feedback selectively disrupts these properties, matching results from inactivation studies in animals. Compared with feedforward temporal prediction models and other unsupervised approaches, the feedback-enabled model better matches neuronal tuning and population-level stimulus representations across multiple visual cortical areas. These findings therefore suggest that inter-areal feedback may be organized to support temporal prediction in the brain.