Searching through functional space reveals distributed visual, auditory, and semantic coding in the human brain

The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.Author summary: There are two classical views about how the mind is organized in the brain. Early phrenology and neurophysiology and later neuropsychology argued that brain regions are specialized for certain functions of the mind. Older behavioral neuroscience and more recent neural network modeling and pattern classification instead argued against a one-to-one mapping, and rather that functions of the mind are distributed across multiple brain regions. Although there is considerable evidence for both perspectives in modern cognitive neuroscience, we hypothesize that the degree to which functions are distributed has been underestimated because of biases in prior work that favored finding specialized regions. Our novel machine learning approach, functional searchlight, reveals that features of a movie extracted with three different types of computational model and object representations are more widely distributed in the brain than suggested by current methods. Moreover, these distributed representations carry more movie content than could previously be decoded from the brain. This suggests a better way to conduct model-based analysis of brain data and provides a more solid basis on which to evaluate and refine theoretical models.