IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v12y2021i1d10.1038_s41467-021-25427-4.html
   My bibliography  Save this article

Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

Author

Listed:
  • Florian Stelzer

    (Technische Universität Berlin
    Humboldt-Universität zu Berlin
    University of Tartu)

  • André Röhm

    (IFISC (UIB-CSIC), Campus Universitat de les Illes Baleares)

  • Raul Vicente

    (University of Tartu)

  • Ingo Fischer

    (IFISC (UIB-CSIC), Campus Universitat de les Illes Baleares)

  • Serhiy Yanchuk

    (Technische Universität Berlin)

Abstract

Deep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Suggested Citation

  • Florian Stelzer & André Röhm & Raul Vicente & Ingo Fischer & Serhiy Yanchuk, 2021. "Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    • Handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-25427-4
    DOI: 10.1038/s41467-021-25427-4
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-021-25427-4
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-021-25427-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Semenov, Vladimir V. & Bukh, Andrei V. & Semenova, Nadezhda, 2023. "Delay-induced self-oscillation excitation in the Fitzhugh–Nagumo model: Regular and chaotic dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 172(C).
    2. Lina Jaurigue & Kathy Lüdge, 2022. "Connecting reservoir computing with statistical forecasting and deep neural networks," Nature Communications, Nature, vol. 13(1), pages 1-3, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-25427-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.