Memory capacity expansion in a sine activated Hopfield neural network

It has been documented that using non-monotonic activation functions in neural networks can expand their memory capacity, and their performance is superior to that of traditional ones that only use monotonic activation functions. This study proposes a three-neuron sine activated Hopfield neural network (SA-HNN) without self-connections, and explores its multi-stable dynamics induced by the sine activation function, demonstrating the expansion of memory capacity. The boundedness and ultimate boundedness are proved, and the equilibria with stability are analyzed. Using numerical methods, the synaptic weights related bifurcation behaviors under different initial states are investigated. The results show that SA-HNN can present up to 14 heterogeneous coexistence attractors, far more than the number found in HNNs using other activation functions, manifesting that the sine activation function indeed expands the memory capacity of HNNs. Finally, an analog circuit is designed and hardware experiments are performed to acquire various coexistence attractors, thereby validating the numerical results.