The beneficial role of noises for disentanglement tasks in modular Hebbian networks

An assembly of neural networks, where both intra- and inter-network couplings are of Hebbian nature and built on the same set of patterns {ξμ}μ=1,…,K, has recently been shown to be able to accomplish a disentanglement task: when inputted with a mixture of patterns, say sign(ξ1+ξ2+ξ3), it can return the single patterns making up the mixture, say {ξ1,ξ2,ξ3}. In this work we generalize its treatment by dropping the hypothesis of perfect accessibility to the patterns, that is, we revise the interaction strengths by adopting a supervised Hebbian rule, based on the availability of M corrupted copies {ηaμ}a=1,…,M for each pattern ξμ with μ=1,…,K. We perform a statistical mechanics investigation and we reach an explicit expression of the system free-energy (under the replica-symmetry ansatz) as a function of the system control parameters, namely the load, the temperature, the dataset entropy, and the ratio between inter- and intra-network interactions. Building on this knowledge and focusing on the low-load regime, we determine operating regimes and optimal values for the control parameters, and we discuss possible strategies for an efficient use of the dataset, specifically, whether it is more convenient to employ the whole dataset for each network ( making the disentangled configuration more attractive) or to split it among the networks (making the spurious configurations less stable).