Author
Listed:
- Centonze, Martino Salomone
- Kanter, Ido
- Barra, Adriano
Abstract
We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative – involving couples of patterns – and it is achieved by reverberating the information depicted from the examples through the layers of the network. By adapting Guerra’s interpolation technique, we provide a full statistical mechanical picture of supervised and unsupervised learning processes (at the replica symmetric level of description) obtaining analytically phase diagrams, thresholds for learning, a picture of the ground-state in plain agreement with Monte Carlo simulations and signal-to-noise outcomes. In the large dataset limit, the Kosko storage prescription as well as its statistical mechanical picture provided by Kurchan, Peliti, and Saber in the eighties is fully recovered. Computational advantages in dealing with information reverberation, rather than storage, are discussed for natural test cases. In particular, we show how this network admits an integral representation in terms of two coupled restricted Boltzmann machines, whose hidden layers are entirely built of by grand-mother neurons, to prove that by coupling solely these grand-mother neurons we can correlate the patterns they are related to: it is thus possible to recover Pavlov’s Classical Conditioning by adding just one synapse among the correct grand-mother neurons (hence saving an extensive number of these links for further information storage w.r.t. the classical auto-associative setting). Likewise, it is enough to equip the BAM’s synaptic coupling with its transpose matrix (and alternate among these two, e.g., by introducing a clock) to let these networks be able count sequences of patterns (still preserving Detailed Balance, at difference w.r.t. the auto-associative case).
Suggested Citation
Centonze, Martino Salomone & Kanter, Ido & Barra, Adriano, 2024.
"Statistical mechanics of learning via reverberation in bidirectional associative memories,"
Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 637(C).
Handle:
RePEc:eee:phsmap:v:637:y:2024:i:c:s0378437124000207
DOI: 10.1016/j.physa.2024.129512
Download full text from publisher
As the access to this document is restricted, you may want to search for a different version of it.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:637:y:2024:i:c:s0378437124000207. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
We have no bibliographic references for this item. You can help adding them by using this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .
Please note that corrections may take a couple of weeks to filter through
the various RePEc services.