Predictive coding explains asymmetric connectivity in the brain: A neural network study

Seminal frameworks of predictive coding propose a hierarchy of generative modules, each attempting to infer the neural representation of the module one level below; the predictions are carried by top-down feedback projections, while the predictive error is propagated by reciprocal forward pathways. Such symmetric feedback connections support visual processing of noisy stimuli in computational models. However, neurophysiological studies have yielded evidence of asymmetric cortical feedback connections. We investigated the contribution of neural feedback in visual processing for computing grasp parameters, by utilizing convolutional neural network models that had been augmented with predictive feedback and were trained to compute grasp positions for real-world objects. After establishing an ameliorative effect of symmetric feedback on grasp detection performance when evaluated on noisy stimuli, we characterized the performance effects of asymmetric feedback, similar to that observed in the cortex. Specifically, we tested model variants extended with short-, medium-, long- and longer-range feedback connections (i) originating at the same source layer or (ii) terminating at the same target layer. We found that the performance-enhancing effect of predictive coding under adverse conditions was optimal for medium-range asymmetric feedback. Moreover, this effect was most prominent when medium-range feedback originated at a level of representational abstraction that was proximal to the input layer, in contrast to more distal layers. To conclude, our simulations show that introducing biologically realistic asymmetric predictive feedback improves model robustness to noisy visual stimuli in a neural network model optimized for grasp detection.Author summary: Everyday vision is noisy: objects can be blurred, partly hidden, or poorly illuminated. Predictive coding is a commonly held idea in neuroscience, which proposes that higher brain areas send feedback to lower areas to “clean up” sensory information. But how far should these feedback signals travel, and from which level should they originate? We studied this question with deep neural networks that plan how to grasp objects from noisy images. We added feedback connections to the networks and varied two factors independently: the path length of feedback and the level of abstraction (information richness) of its source. Across varying degrees of image noise, medium-range feedback from relatively early stages of the network produced the most reliable improvements. These results suggest that intermediate visual stages in the brain may be well placed to send predictive signals that stabilize visual representations for computing grasp parameters. Beyond informing neuroscience, our findings provide design principles for making machine perception and future robotic grasping systems more robust to noisy inputs.