IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-42901-3.html
   My bibliography  Save this article

Realizing a deep reinforcement learning agent for real-time quantum feedback

Author

Listed:
  • Kevin Reuer

    (ETH Zurich
    ETH Zurich)

  • Jonas Landgraf

    (Max Planck Institute for the Science of Light
    University of Erlangen-Nuremberg)

  • Thomas Fösel

    (Max Planck Institute for the Science of Light
    University of Erlangen-Nuremberg)

  • James O’Sullivan

    (ETH Zurich
    ETH Zurich)

  • Liberto Beltrán

    (ETH Zurich
    ETH Zurich)

  • Abdulkadir Akin

    (ETH Zurich
    ETH Zurich)

  • Graham J. Norris

    (ETH Zurich
    ETH Zurich)

  • Ants Remm

    (ETH Zurich
    ETH Zurich)

  • Michael Kerschbaum

    (ETH Zurich
    ETH Zurich)

  • Jean-Claude Besse

    (ETH Zurich
    ETH Zurich)

  • Florian Marquardt

    (Max Planck Institute for the Science of Light
    University of Erlangen-Nuremberg)

  • Andreas Wallraff

    (ETH Zurich
    ETH Zurich)

  • Christopher Eichler

    (ETH Zurich
    University of Erlangen-Nuremberg)

Abstract

Realizing the full potential of quantum technologies requires precise real-time control on time scales much shorter than the coherence time. Model-free reinforcement learning promises to discover efficient feedback strategies from scratch without relying on a description of the quantum system. However, developing and training a reinforcement learning agent able to operate in real-time using feedback has been an open challenge. Here, we have implemented such an agent for a single qubit as a sub-microsecond-latency neural network on a field-programmable gate array (FPGA). We demonstrate its use to efficiently initialize a superconducting qubit and train the agent based solely on measurements. Our work is a first step towards adoption of reinforcement learning for the control of quantum devices and more generally any physical device requiring low-latency feedback.

Suggested Citation

  • Kevin Reuer & Jonas Landgraf & Thomas Fösel & James O’Sullivan & Liberto Beltrán & Abdulkadir Akin & Graham J. Norris & Ants Remm & Michael Kerschbaum & Jean-Claude Besse & Florian Marquardt & Andreas, 2023. "Realizing a deep reinforcement learning agent for real-time quantum feedback," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42901-3
    DOI: 10.1038/s41467-023-42901-3
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-42901-3
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-42901-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. V. Negnevitsky & M. Marinelli & K. K. Mehta & H.-Y. Lo & C. Flühmann & J. P. Home, 2018. "Repeated multi-qubit readout and feedback with a mixed-species trapped-ion register," Nature, Nature, vol. 563(7732), pages 527-531, November.
    2. L. Steffen & Y. Salathe & M. Oppliger & P. Kurpiers & M. Baur & C. Lang & C. Eichler & G. Puebla-Hellmann & A. Fedorov & A. Wallraff, 2013. "Deterministic quantum teleportation with feed-forward in a solid state system," Nature, Nature, vol. 500(7462), pages 319-322, August.
    3. Kevin S. Chou & Jacob Z. Blumoff & Christopher S. Wang & Philip C. Reinhold & Christopher J. Axline & Yvonne Y. Gao & L. Frunzio & M. H. Devoret & Liang Jiang & R. J. Schoelkopf, 2018. "Deterministic teleportation of a quantum gate between two logical qubits," Nature, Nature, vol. 561(7723), pages 368-373, September.
    4. Nissim Ofek & Andrei Petrenko & Reinier Heeres & Philip Reinhold & Zaki Leghtas & Brian Vlastakis & Yehan Liu & Luigi Frunzio & S. M. Girvin & L. Jiang & Mazyar Mirrahimi & M. H. Devoret & R. J. Schoe, 2016. "Extending the lifetime of a quantum bit with error correction in superconducting circuits," Nature, Nature, vol. 536(7617), pages 441-445, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. William P. Livingston & Machiel S. Blok & Emmanuel Flurin & Justin Dressel & Andrew N. Jordan & Irfan Siddiqi, 2022. "Experimental demonstration of continuous quantum error correction," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    2. X. L. He & Yong Lu & D. Q. Bao & Hang Xue & W. B. Jiang & Z. Wang & A. F. Roudsari & Per Delsing & J. S. Tsai & Z. R. Lin, 2023. "Fast generation of Schrödinger cat states using a Kerr-tunable superconducting resonator," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    3. Costa, A.C.S. & Beims, M.W. & Angelo, R.M., 2016. "Generalized discord, entanglement, Einstein–Podolsky–Rosen steering, and Bell nonlocality in two-qubit systems under (non-)Markovian channels: Hierarchy of quantum resources and chronology of deaths a," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 461(C), pages 469-479.
    4. W. Wang & Z.-J. Chen & X. Liu & W. Cai & Y. Ma & X. Mu & X. Pan & Z. Hua & L. Hu & Y. Xu & H. Wang & Y. P. Song & X.-B. Zou & C.-L. Zou & L. Sun, 2022. "Quantum-enhanced radiometry via approximate quantum error correction," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    5. Ziqian Li & Tanay Roy & David Rodríguez Pérez & Kan-Heng Lee & Eliot Kapit & David I. Schuster, 2024. "Autonomous error correction of a single logical qubit using two transmons," Nature Communications, Nature, vol. 15(1), pages 1-6, December.
    6. T. Brown & E. Doucet & D. Ristè & G. Ribeill & K. Cicak & J. Aumentado & R. Simmonds & L. Govia & A. Kamal & L. Ranzani, 2022. "Trade off-free entanglement stabilization in a superconducting qutrit-qubit system," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    7. L. Feng & Y.-Y. Huang & Y.-K. Wu & W.-X. Guo & J.-Y. Ma & H.-X. Yang & L. Zhang & Y. Wang & C.-X. Huang & C. Zhang & L. Yao & B.-X. Qi & Y.-F. Pu & Z.-C. Zhou & L.-M. Duan, 2024. "Realization of a crosstalk-avoided quantum network node using dual-type qubits of the same ion species," Nature Communications, Nature, vol. 15(1), pages 1-7, December.
    8. Santos, Jonas F.G. & Vieira, Carlos H.S. & Dieguez, Pedro R., 2021. "Negativity-mutual information conversion and coherence in two-coupled harmonic oscillators," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 579(C).
    9. Jie Zhao & Hao Jeng & Lorcán O. Conlon & Spyros Tserkis & Biveen Shajilal & Kui Liu & Timothy C. Ralph & Syed M. Assad & Ping Koy Lam, 2023. "Enhancing quantum teleportation efficacy with noiseless linear amplification," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    10. Axel M. Eriksson & Théo Sépulcre & Mikael Kervinen & Timo Hillmann & Marina Kudra & Simon Dupouy & Yong Lu & Maryam Khanahmadi & Jiaying Yang & Claudia Castillo-Moreno & Per Delsing & Simone Gasparine, 2024. "Universal control of a bosonic mode via drive-activated native cubic interactions," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    11. Sergei Slussarenko & Morgan M. Weston & Lynden K. Shalm & Varun B. Verma & Sae-Woo Nam & Sacha Kocsis & Timothy C. Ralph & Geoff J. Pryde, 2022. "Quantum channel correction outperforming direct transmission," Nature Communications, Nature, vol. 13(1), pages 1-6, December.
    12. Aleksander Kubica & Michael Vasmer, 2022. "Single-shot quantum error correction with the three-dimensional subsystem toric code," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    13. Eric Hyyppä & Suman Kundu & Chun Fai Chan & András Gunyhó & Juho Hotari & David Janzso & Kristinn Juliusson & Olavi Kiuru & Janne Kotilahti & Alessandro Landra & Wei Liu & Fabian Marxer & Akseli Mäkin, 2022. "Unimon qubit," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    14. Yao Lu & Aniket Maiti & John W. O. Garmon & Suhas Ganjam & Yaxing Zhang & Jahan Claes & Luigi Frunzio & Steven M. Girvin & Robert J. Schoelkopf, 2023. "High-fidelity parametric beamsplitting with a parity-protected converter," Nature Communications, Nature, vol. 14(1), pages 1-11, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42901-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.