- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Quantum feedback control with a transformer neural network architecture

Pranav Vaidhyanathan1, Florian Marquardt2,3, Mark T. Mitchison4,5,*, and Natalia Ares1,†

  • *Contact author: mark.mitchison@kcl.ac.uk
  • †Contact author: natalia.ares@eng.ox.ac.uk
PDF

Phys. Rev. Research 8, L012043 – Published 24 February, 2026

DOI: https://doi.org/10.1103/m429-jy1j

Abstract

Attention-based neural networks such as transformers have revolutionized various fields such as natural language processing, genomics, and vision. Here, we demonstrate the use of transformers for quantum feedback control through both a supervised and a reinforcement learning approach. In particular, due to the transformer's ability to capture long-range temporal correlations and training efficiency, we show that it can surpass some of the limitations of previous control approaches, e.g., those based on recurrent neural networks trained using a similar approach or policy-based reinforcement learning. We numerically show, for the example of state stabilization of a two-level system, that our bespoke transformer architecture can achieve near-unit fidelity to a target state in a short time even in the presence of inefficient measurement and Hamiltonian perturbations that were not included in the training set as well as the control of non-Markovian systems. We also demonstrate that our transformer can perform energy minimization of nonintegrable many-body quantum systems when trained for reinforcement learning tasks. Our approach can be used for quantum error correction, fast control of quantum states in the presence of colored noise, as well as real-time tuning, and characterization of quantum devices.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (47)

  1. J. Zhang, Y.-X. Liu, R.-B. Wu, K. Jacobs, and F. Nori, Quantum feedback: Theory, experiments, and applications, Phys. Rep. 679, 1 (2017).
  2. M. Puviani, S. Borah, R. Zen, J. Olle, and F. Marquardt, Non-Markovian feedback for optimized quantum error correction, Phys. Rev. Lett. 134, 020601 (2025).
  3. H. M. Wiseman, S. Mancini, and J. Wang, Bayesian feedback versus Markovian feedback in a two-level atom, Phys. Rev. A 66, 013807 (2002).
  4. H. M. Wiseman and G. J. Milburn, Quantum Measurement and Control, 1st ed. (Cambridge University Press, Cambridge, UK, 2009).
  5. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement learning with neural networks for quantum feedback, Phys. Rev. X 8, 031084 (2018).
  6. Z. T. Wang, Y. Ashida, and M. Ueda, Deep reinforcement learning control of quantum cartpoles, Phys. Rev. Lett. 125, 100401 (2020).
  7. V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios, and M. H. Devoret, Model-free quantum control with reinforcement learning, Phys. Rev. X 12, 011059 (2022).
  8. K. Reuer, J. Landgraf, T. Fösel, J. O’Sullivan, L. Beltrán, A. Akin, G. J. Norris, A. Remm, M. Kerschbaum, J.-C. Besse, F. Marquardt, A. Wallraff, and C. Eichler, Realizing a deep reinforcement learning agent for real-time quantum feedback, Nat. Commun. 14, 7138 (2023).
  9. S. Borah, B. Sarma, M. Kewming, G. J. Milburn, and J. Twamley, Measurement-based feedback quantum control with deep reinforcement learning for a double-well nonlinear potential, Phys. Rev. Lett. 127, 190403 (2021).
  10. R. Porotti, V. Peano, and F. Marquardt, Gradient-ascent pulse engineering with feedback, PRX Quantum 4, 030305 (2023).
  11. Y. Bengio, P. Simard, and P. Frasconi, Learning long-term dependencies with gradient descent is difficult, IEEE Trans. Neural Netw. 5, 157 (1994).
  12. E. Genois, J. A. Gross, A. Di Paolo, N. J. Stevenson, G. Koolstra, A. Hashim, I. Siddiqi, and A. Blais, Quantum-tailored machine-learning characterization of a superconducting qubit, PRX Quantum 2, 040355 (2021).
  13. H.-P. Breuer, E.-M. Laine, J. Piilo, and B. Vacchini, Colloquium: Non-Markovian dynamics in open quantum systems, Rev. Mod. Phys. 88, 021002 (2016).
  14. G. T. Landi, M. J. Kewming, M. T. Mitchison, and P. P. Potts, Current fluctuations in open quantum systems: Bridging the gap between quantum continuous measurements and full counting statistics, PRX Quantum 5, 020201 (2024).
  15. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems (Curran Associates, Inc., 2017), Vol. 30, pp. 6000–6010.
  16. U. Kamath, K. Graham, and W. Emara, Transformers for Machine Learning: A Deep Dive (Chapman & Hall/CRC, Boca Raton, Florida, USA, 2022).
  17. M. Janner, Q. Li, and S. Levine, Offline reinforcement learning as one big sequence modeling problem, in NIPS'21: Proceedings of the 35th International Conference on Neural Information Processing Systems (Curran Associates, Inc., 2021), Vol. 34, pp. 1273–1286.
  18. L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, Decision transformer: Reinforcement learning via sequence modeling, in Advances in Neural Information Processing Systems (NeurIPS, 2021), Vol. 34, pp. 15084–15097.
  19. K. Jacobs and D. A. Steck, A straightforward introduction to continuous quantum measurement, Contemp. Phys. 47, 279 (2006).
  20. H. M. Wiseman and G. J. Milburn, Quantum theory of optical feedback via homodyne detection, Phys. Rev. Lett. 70, 548 (1993).
  21. H. M. Wiseman, Quantum theory of continuous feedback, Phys. Rev. A 49, 2133 (1994).
  22. A. N. Korotkov, Continuous quantum measurement of a double dot, Phys. Rev. B 60, 5737 (1999).
  23. H.-S. Goan, G. J. Milburn, H. M. Wiseman, and H. Bi Sun, Continuous quantum measurement of two coupled quantum dots using a point contact: A quantum trajectory approach, Phys. Rev. B 63, 125326 (2001).
  24. Y. Jiang, X. Wang, L. Martin, and K. B. Whaley, Optimality of feedback control for qubit purification under inefficient measurement, Phys. Rev. A 102, 022612 (2020).
  25. J. Wang and H. M. Wiseman, Feedback-stabilization of an arbitrary pure state of a two-level atom, Phys. Rev. A 64, 063810 (2001).
  26. M. T. Mitchison, J. Goold, and J. Prior, Charging a quantum battery with linear feedback control, Quantum 5, 500 (2021).
  27. S. Zhang, L. S. Martin, and K. B. Whaley, Locally optimal measurement-based quantum feedback with application to multiqubit entanglement generation, Phys. Rev. A 102, 062418 (2020).
  28. See Supplemental Material at http://link.aps.org/supplemental/10.1103/m429-jy1j for more examples and details on the model training, architecture, and performance.
  29. J. Johansson, P. Nation, and F. Nori, QuTiP 2: A Python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 184, 1234 (2013).
  30. C.-F. Li, G.-C. Guo, and J. Piilo, Non-Markovian quantum dynamics: What is it good for?, Europhys. Lett. 128, 30001 (2020).
  31. A. Blais, R.-S. Huang, A. Wallraff, S. M. Girvin, and R. J. Schoelkopf, Cavity quantum electrodynamics for superconducting electrical circuits: An architecture for quantum computation, Phys. Rev. A 69, 062320 (2004).
  32. H.-P. Breuer and F. Petruccione, The Theory of Open Quantum Systems (Oxford University Press, Oxford, 2002).
  33. J. Iles-Smith, N. Lambert, and A. Nazir, Environmental dynamics, correlations, and the emergence of noncanonical equilibrium states in open quantum systems, Phys. Rev. A 90, 032114 (2014).
  34. D. Tamascelli, A. Smirne, S. F. Huelga, and M. B. Plenio, Nonperturbative treatment of non-Markovian dynamics of open quantum systems, Phys. Rev. Lett. 120, 030402 (2018).
  35. M. P. Woods, R. Groux, A. W. Chin, S. F. Huelga, and M. B. Plenio, Mappings of open quantum systems onto chain representations and Markovian embeddings, J. Math. Phys. 55, 032101 (2014).
  36. K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (Association for Computational Linguistics, Doha, Qatar, 2014), pp. 1724–1734.
  37. R. Pascanu, T. Mikolov, and Y. Bengio, On the difficulty of training recurrent neural networks, in Proceedings of the 30th International Conference on Machine Learning, edited by S. Dasgupta and D. McAllester (PMLR, Atlanta, GA, 2013), Vol. 28, pp. 1310–1318.
  38. F. Metz and M. Bukov, Self-correcting quantum many-body control using reinforcement learning with tensor networks, Nat. Mach. Intell. 5, 780 (2023).
  39. Y.-H. Zhang and M. Di Ventra, Transformer quantum state: A multipurpose model for quantum many-body problems, Phys. Rev. B 107, 075147 (2023).
  40. M. Kolodrubetz, D. Sels, P. Mehta, and A. Polkovnikov, Geometry and non-adiabatic response in quantum and classical systems, Phys. Rep. 697, 1 (2017).
  41. D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Shortcuts to adiabaticity: Concepts, methods, and applications, Rev. Mod. Phys. 91, 045001 (2019).
  42. H. Kim and D. A. Huse, Ballistic spreading of entanglement in a diffusive nonintegrable system, Phys. Rev. Lett. 111, 127205 (2013).
  43. J. F. Rodriguez-Nieva, C. Jonay, and V. Khemani, Quantifying quantum chaos through microcanonical distributions of entanglement, Phys. Rev. X 14, 031014 (2024).
  44. Q. Zheng, A. Zhang, and A. Grover, Online decision transformer, in Proceedings of Machine Learning Research (Baltimore, Maryland, USA, 2022), pp. 27042–27059.
  45. L. E. H. Rodríguez and A. A. Kananenka, A short trajectory is all you need: A transformer-based model for long-time dissipative quantum dynamics, J. Chem. Phys. 161, 171101 (2024).
  46. http://dx.doi.org/10.5281/zenodo.22558.
  47. Code for the network architecture and data generation, GitHub (2025), https://github.com/pranavjv/transformerfeedbackcontrol.

Outline

Information

Phys. Rev. Research 8, L012043– Published 24 February, 2026
  • Received 21 December 2024
  • Accepted 1 November 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation