Population sparseness determines strength of Hebbian plasticity for maximal memory lifetime in associative networks

The brain can efficiently learn and form memories based on limited exposure to stimuli, often even in single trials. Two key factors are believed to support this ability: large synaptic plasticity to strongly encode new memories; and sparse coding, leading to low overlap between memory representations and to small interference. Therefore, increased sparseness can also improve memory capacity. However, it is not well understood how the strength of plasticity of synapses affects capacity. Here, we analyze the combined impact of population sparseness and strength of plasticity on memory capacity. Specifically, we explore how the strength of plasticity that maximizes capacity depends on the sparseness of the neural code. To this end, we study a feedforward network with Hebbian and homeostatic plasticity and a two-state synapse model. The network learns to associate sparse binary input-output pattern pairs. The strength of plasticity is modeled as the probability of synaptic changes. Our results are based on both network simulations and an analytical theory, predicting the expected memory capacity in dependence on strength of plasticity and population sparseness. For both perfect and noisy input patterns, we find that the optimal strength of plasticity increases with increasing pattern sparseness and that this effect is more pronounced for input than for output sparseness. Interestingly, the optimal strength of plasticity remains the same across different network sizes if the number of active units in an input pattern is constant. While the memory capacity obtained at the optimal strength of plasticity increases monotonically with output sparseness, its dependence on input sparseness is non-monotonic. Overall, we provide the first detailed investigation of the interactions between population sparseness, strength of plasticity, and memory capacity. Our findings suggest that differences in sparseness between brain regions may underlie observed differences in how strongly these regions adapt and how quickly they learn.Author summary: The brain can efficiently learn and form memories based on limited exposure to stimuli. This ability is believed to be supported by the way neural circuits encode and organize information about the external world. High population sparseness refers to the phenomenon in which, in many brain regions, only a small subset of neurons is active at any given time or in response to a particular stimulus. Such sparse codes are believed to reduce overlap between representations and minimize interference and can thus enhance the storage capacity of a network. Here, we investigate the relation of population sparseness and the strength of plasticity when forming input-output associations in a network model with Hebbian learning. We find that the strength of plasticity that yields the maximal capacity increases with increasing sparseness. The maximal capacity increases for sparser input and output representations, but in the first case only up to a certain point before sparseness becomes destructive. These findings could contribute to explaining why some brain regions learn with higher plasticity than others.