Stochastic activity in low-rank recurrent neural networks

The geometrical and statistical properties of brain activity depend on the way neurons connect to form recurrent circuits. However, the link between connectivity structure and emergent activity remains incompletely understood. We investigate this relationship in recurrent neural networks with additive stochastic inputs. We assume that the synaptic connectivity can be expressed in a low-rank form, parameterized by a handful of connectivity vectors, and examine how the geometry of emergent activity relates to these vectors. Our findings reveal that this relationship critically depends on the dimensionality of the external stochastic inputs. When inputs are low-dimensional, activity remains low-dimensional, and recurrent dynamics influence it within a subspace spanned by a subset of the connectivity vectors, with dimensionality equal to the rank of the connectivity matrix. In contrast, when inputs are high-dimensional, activity also becomes potentially high-dimensional. The contribution of recurrent dynamics is apparent within a subspace spanned by the totality of the connectivity vectors, with dimensionality equal to twice the rank of the connectivity matrix. Applying our formalism to excitatory-inhibitory networks, we discuss how the input configuration also plays a crucial role in determining the amount of amplification generated by non-normal dynamics. Our work provides a foundation for studying activity in structured brain circuits under realistic noise conditions, and offers a framework for interpreting stochastic models inferred from experimental data.Author summary: Low-rank recurrent networks have recently emerged as a mathematically tractable framework for studying the relationship between connectivity and activity in biological and artificial neural circuits. Those models naturally produce low-dimensional activity patterns, consistent with brain recordings during cognitive tasks. However, they have so far been studied in settings with highly simplified external inputs, limiting their applicability to biologically relevant scenarios. In this work, we investigate the dynamics of low-rank networks driven by noisy inputs with varying geometries. We find that when inputs are high-dimensional, the statistics and geometry of the resulting activity differ markedly from previous descriptions. In particular, activity can become high-dimensional. Among the many dimensions it spans, those shaped by recurrent interactions are both more numerous and structurally distinct compared to those in networks receiving simpler inputs. While some of these directions encode input amplification by recurrent connectivity, others reflect input suppression. By extending the low-rank framework to more realistic settings, our work opens new avenues for applications in data analysis, and in modeling learning and variability in cortical circuits.