How distinct sources of nuisance variability in natural images and scenes limit human stereopsis

Stimulus variability—a form of nuisance variability—is a primary source of perceptual uncertainty in everyday natural tasks. How do different properties of natural images and scenes contribute to this uncertainty? Using binocular disparity as a model system, we report a systematic investigation of how various forms of natural stimulus variability impact performance in a stereo-depth discrimination task. With stimuli sampled from a stereo-image database of real-world scenes having pixel-by-pixel ground-truth distance data, three human observers completed two closely related double-pass psychophysical experiments. In the two experiments, each human observer responded twice to ten thousand unique trials, in which twenty thousand unique stimuli were presented. New analytical methods reveal, from this data, the specific and nearly dissociable effects of two distinct sources of natural stimulus variability—variation in luminance-contrast patterns and variation in local-depth structure—on discrimination performance, as well as the relative importance of stimulus-driven-variability and internal-noise in determining performance limits. Between-observer analyses show that both stimulus-driven sources of uncertainty are responsible for a large proportion of total variance, have strikingly similar effects on different people, and—surprisingly—make stimulus-by-stimulus responses more predictable (not less). The consistency across observers raises the intriguing prospect that image-computable models can make reasonably accurate performance predictions in natural viewing. Overall, the findings provide a rich picture of stimulus factors that contribute to human perceptual performance in natural scenes. The approach should have broad application to other animal models and other sensory-perceptual tasks with natural or naturalistic stimuli.Author summary: Linking properties of the external world, and of sensory stimuli, to how neurons and animals respond has proven an important approach to understanding how the brain works. Much is known about how nervous systems respond to simple stimuli. Less is known about how systems respond to real-world stimuli. A major challenge is to present stimuli in an experimental setting that reflect important aspects of the variability that is present in natural viewing, while maintaining the rigor and interpretability that is necessary for drawing scientific conclusions about what drives and limits perceptual performance. Using a high-fidelity database of natural images and scenes, we conducted two human stereo-depth discrimination experiments and analyzed the data with a newly developed method that reveals how distinct features of natural scenes and images impact performance. Results show that stimulus-by-stimulus variation has highly consistent effects on different people. The approach should have broad application to other animal models and other sensory-perceptual tasks.