Statistical regularities in natural scenes that support figure-ground segregation by neural populations

Differentiating objects, people, and animals from their surroundings is a key visual function, referred to as figure-ground segregation. Psychological research has established that humans use diverse visual features such as shape, texture, motion, and distance to identify figures. However, our understanding of the neural computations supporting figure-ground segregation remains incomplete. Recent neurophysiological observations in cortical area MT of primates – a region important for motion and depth processing – suggest that neurons in this area favor visual features that intuitively map onto figures, such as faster motion and closer distances. Inspired by these new observations, here we test the hypothesis that figures in natural scenes contain statistical regularities in motion and distance detectable at the scale of neuronal receptive fields. We combined statistical measurements of motion and distance from natural scenes with figure-ground annotations and simulations of receptive field inputs. Within simulated receptive fields, inputs corresponding to figures tended to move faster and more coherently, and tended to be nearer in distance, than the nearby ground. Our simulation predicts that the statistical regularities associated with figure motion increase notably with retinal eccentricity, while the distance statistics do not. Lastly, we implemented a simple neural population model illustrating how MT response properties, in combination with these statistics, can prioritize the representation of visual features associated with figures. These results enrich our understanding of the computations supporting figure-ground segregation, provide a normative account for recent neurophysiological observations, and contribute to converging lines of evidence that the brain exploits natural statistics to prioritize behaviorally-relevant information.Author summary: For many visually-guided activities, animals must first segment out relevant objects, people, and animals from the surroundings. This process is called figure-ground segregation. We tested the hypothesis that recently discovered neuronal response properties reflect a neural code that prioritizes visual features typical of figures in natural scenes – specifically, their motion and distance characteristics. Our approach employed natural scene statistics, precise image annotations, and behavioral/neural simulations. The findings support our working hypothesis, make new and testable predictions for neurophysiology, and illustrate the diverse ways that the statistics of natural stimuli can support not only efficient neural codes, but also neural codes that boost behaviorally-relevant information for tasks like figure-ground segregation.