Tensor Analysis Reveals Distinct Population Structure that Parallels the Different Computational Roles of Areas M1 and V1

Cortical firing rates frequently display elaborate and heterogeneous temporal structure. One often wishes to compute quantitative summaries of such structure—a basic example is the frequency spectrum—and compare with model-based predictions. The advent of large-scale population recordings affords the opportunity to do so in new ways, with the hope of distinguishing between potential explanations for why responses vary with time. We introduce a method that assesses a basic but previously unexplored form of population-level structure: when data contain responses across multiple neurons, conditions, and times, they are naturally expressed as a third-order tensor. We examined tensor structure for multiple datasets from primary visual cortex (V1) and primary motor cortex (M1). All V1 datasets were ‘simplest’ (there were relatively few degrees of freedom) along the neuron mode, while all M1 datasets were simplest along the condition mode. These differences could not be inferred from surface-level response features. Formal considerations suggest why tensor structure might differ across modes. For idealized linear models, structure is simplest across the neuron mode when responses reflect external variables, and simplest across the condition mode when responses reflect population dynamics. This same pattern was present for existing models that seek to explain motor cortex responses. Critically, only dynamical models displayed tensor structure that agreed with the empirical M1 data. These results illustrate that tensor structure is a basic feature of the data. For M1 the tensor structure was compatible with only a subset of existing models.Author Summary: Neuroscientists commonly measure the time-varying activity of neurons in the brain. Early studies explored how such activity directly encodes sensory stimuli. Since then neural responses have also been found to encode abstract parameters such as expected reward. Yet not all aspects of neural activity directly encode identifiable parameters: patterns of activity sometimes reflect the evolution of underlying internal computations, and may be only obliquely related to specific parameters. For example, it remains debated whether cortical activity during movement relates to parameters such as reach velocity, to parameters such as muscle activity, or to underlying computations that culminate in the production of muscle activity. To address this question we exploited an unexpected fact. When activity directly encodes a parameter it tends to be mathematically simple in a very particular way. When activity reflects the evolution of a computation being performed by the network, it tends to be mathematically simple in a different way. We found that responses in a visual area were simple in the first way, consistent with encoding of parameters. We found that responses in a motor area were simple in the second way, consistent with participation in the underlying computations that culminate in movement.