- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Critical states preparation in light-matter systems with deep reinforcement learning

Jia-Wen Yu1,2, Yiming Yu1,2, Ke-Xiong Yan1,2, Jun-Hao Lin1,2, Jie Song3, Ye-Hong Chen1,2,4,*, and Yan Xia1,2,†

  • 1Department of Physics, Fuzhou University, Fuzhou 350116, China
  • 2Fujian Key Laboratory of Quantum Information and Quantum Optics, Fuzhou University, Fuzhou 350116, China
  • 3Department of Physics, Harbin Institute of Technology, Harbin 150001, China
  • 4Quantum Information Physics Theory Research Team, Center for Quantum Computing, RIKEN, Wako-shi, Saitama 351-0198, Japan

  • *Contact author: yehong.chen@fzu.edu.cn
  • †Contact author: xia-208@163.com
PDF

Phys. Rev. Research 8, 033063 – Published 15 July, 2026

DOI: https://doi.org/10.1103/34yc-wj82

Abstract

The fast and efficient preparation of quantum critical states is a challenging yet crucial task for various quantum technologies. This difficulty is most particularly for systems near a quantum phase transition, where the closure of the energy gap fundamentally limits the timescale of adiabatic processes and thus precludes rapid state preparation. We propose a framework using deep reinforcement learning (DRL) to rapidly prepare quantum critical states, with broad extendibility to light-matter interaction systems. Specifically, a DRL agent optimizes a set of time-dependent control Hamiltonians to drive the system from an initial noncritical state to a target critical state within a finite time and over experimentally accessible parameter ranges. As a concrete application, we focus on the quantum Rabi model. The DRL-optimized time-dependent control Hamiltonian yields a final state with high fidelity (>0.999) to the target critical state. The protocol can be readily extended to other quantum critical systems described by light-matter interaction models, such as quantum Dicke model. This investigation provides a powerful framework for preparing and manipulating quantum critical states.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (68)

  1. M. O. Scully and M. S. Zubairy, Quantum Optics (Cambridge University Press, Cambridge, 1997).
  2. Y.-H. Chen, Y. Qiu, A. Miranowicz, N. Lambert, W. Qin, R. Stassi, Y. Xia, S.-B. Zheng, and F. Nori, Sudden change of the photon output field marks phase transitions in the quantum Rabi model, Commun. Phys. 7, 5 (2024).
  3. D. Rossini and E. Vicari, Coherent and dissipative dynamics at quantum phase transitions, Phys. Rep. 936, 1 (2021).
  4. S. Sachdev, Quantum Phase Transitions, 2nd ed. (Cambridge University Press, Cambridge, 2011).
  5. M.-J. Hwang, R. Puebla, and M. B. Plenio, Quantum phase transition and universal dynamics in the Rabi model, Phys. Rev. Lett. 115, 180404 (2015).
  6. M.-J. Hwang, P. Rabl, and M. B. Plenio, Dissipative phase transition in the open quantum Rabi model, Phys. Rev. A 97, 013825 (2018).
  7. W. H. Zurek, U. Dorner, and P. Zoller, Dynamics of a quantum phase transition, Phys. Rev. Lett. 95, 105701 (2005).
  8. P. Zanardi and N. Paunković, Ground state overlap and quantum phase transitions, Phys. Rev. E 74, 031123 (2006).
  9. A. Carollo, D. Valenti, and B. Spagnolo, Geometry of quantum phase transitions, Phys. Rep. 838, 1 (2020), .
  10. A. Osterloh, L. Amico, G. Falci, and R. Fazio, Scaling of entanglement close to a quantum phase transition, Nature (London) 416, 608 (2002).
  11. G. Vidal, J. I. Latorre, E. Rico, and A. Kitaev, Entanglement in quantum critical phenomena, Phys. Rev. Lett. 90, 227902 (2003).
  12. V. E. Korepin, Universality of entropy scaling in one dimensional gapless models, Phys. Rev. Lett. 92, 096402 (2004).
  13. L. Garbe, M. Bina, A. Keller, M. G. A. Paris, and S. Felicetti, Critical quantum metrology with a finite-component quantum phase transition, Phys. Rev. Lett. 124, 120504 (2020).
  14. I. Frérot and T. Roscilde, Quantum critical metrology, Phys. Rev. Lett. 121, 020402 (2018).
  15. R. Salvia, M. Mehboudi, and M. Perarnau-Llobet, Critical quantum metrology assisted by real-time feedback control, Phys. Rev. Lett. 130, 240803 (2023).
  16. J. A. Vaitkus and A. D. Greentree, Digital three-state adiabatic passage, Phys. Rev. A 87, 063820 (2013).
  17. R. Menchon-Enrich, J. Mompart, and V. Ahufinger, Spatial adiabatic passage processes in sonic crystals with linear defects, Phys. Rev. B 89, 094304 (2014).
  18. Y.-H. Chen, Y. Xia, Q.-C. Wu, B.-H. Huang, and J. Song, Method for constructing shortcuts to adiabaticity by a substitute of counterdiabatic driving terms, Phys. Rev. A 93, 052109 (2016).
  19. L. Innocenti, G. De Chiara, M. Paternostro, and R. Puebla, Ultrafast critical ground state preparation via bang–bang protocols, New J. Phys. 22, 093050 (2020).
  20. S. Bernier and K. Agarwal, Spatiotemporal quenches for efficient critical ground state preparation in the two-dimensional transverse field Ising model, Phys. Rev. B 111, 054311 (2025).
  21. Y. Yamamoto and H. A. Haus, Preparation, measurement and information capacity of optical quantum states, Rev. Mod. Phys. 58, 1001 (1986).
  22. R. van Handel, J. K. Stockton, and H. Mabuchi, Modelling and feedback control design for quantum state preparation, J. Opt. B: Quantum Semiclass. Opt. 7, S179 (2005).
  23. K.-X. Hu, B.-H. Huang, S.-W. Xu, Z.-C. Shi, Y. Xia, and Y.-H. Chen, High-fidelity control of cat-state qubits by polychromatic pulses, Phys. Rev. Appl. 25, 024083 (2026).
  24. D.-S. Li, Y.-H. Kang, Z.-C. Shi, Y. Xiao, Y.-H. Chen, and Y. Xia, Efficient and flexible preparation of photonic noon states in a superconducting system, Phys. Rev. A 113, 012619 (2026).
  25. G.-G. Gu, D.-S. Li, Y.-H. Chen, B.-H. Huang, and Y. Xia, Preparation of high-fidelity entangled cat states with composite pulses, Adv. Quantum Technol. 8, 2400518 (2025).
  26. D. Girolami, How difficult is it to prepare a quantum state? Phys. Rev. Lett. 122, 010505 (2019).
  27. F. Yang, Y.-C. Liu, and L. You, Manipulating photonic quantum states with long-range interactions, Phys. Rev. A 99, 063803 (2019).
  28. M. Xue, X. Li, W. Ye, J.-J. Chen, Z.-F. Xu, and L. You, Controlling atomic spin mixing via multiphoton transitions in a cavity, Phys. Rev. A 106, 033708 (2022).
  29. J.-W. Yu, K.-X. Yan, Y. Qiu, J. Song, Y.-H. Chen, and Y. Xia, Efficient generation of arbitrary photon-number-squeezed light via shortcuts to adiabaticity, Opt. Express 33, 26356 (2025).
  30. Y.-H. Chen, W. Qin, X. Wang, A. Miranowicz, and F. Nori, Shortcuts to adiabaticity for the quantum Rabi model: Efficient generation of giant entangled cat states via parametric amplification, Phys. Rev. Lett. 126, 023602 (2021).
  31. M. Mamaev, L. C. G. Govia, and A. A. Clerk, Dissipative stabilization of entangled cat states using a driven Bose-Hubbard dimer, Quantum 2, 58 (2018).
  32. Y. Zeng, W. Qin, Y.-H. Chen, C. Gneiting, and F. Nori, Neural-network-based design of approximate Gottesman-Kitaev-Preskill code, Phys. Rev. Lett. 134, 060601 (2025).
  33. X. Xu, S. C. Benjamin, and X. Yuan, Variational circuit compiler for quantum error correction, Phys. Rev. Appl. 15, 034068 (2021).
  34. F. A. Cárdenas-López, J. C. Retamal, and X. Chen, Shortcuts to adiabaticity in superconducting circuits for fast multi-partite state generation, Commun. Phys. 6, 167 (2023).
  35. D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Shortcuts to adiabaticity: Concepts, methods, and applications, Rev. Mod. Phys. 91, 045001 (2019).
  36. N. Khaneja, T. Reiss, C. Kehlet, T. Schulte-Herbrüggen, and S. J. Glaser, Optimal control of coupled spin dynamics: Design of NMR pulse sequences by gradient ascent algorithms, J. Magn. Reson. 172, 296 (2005).
  37. S. Machnes, U. Sander, S. J. Glaser, P. de Fouquières, A. Gruslys, S. Schirmer, and T. Schulte-Herbrüggen, Comparing, optimizing, and benchmarking quantum-control algorithms in a unifying programming framework, Phys. Rev. A 84, 022305 (2011).
  38. R. Porotti, V. Peano, and F. Marquardt, Gradient-ascent pulse engineering with feedback, PRX Quantum 4, 030305 (2023).
  39. Y. Zeng, Z.-Y. Zhou, E. Rinaldi, C. Gneiting, and F. Nori, Approximate autonomous quantum error correction with reinforcement learning, Phys. Rev. Lett. 131, 050601 (2023).
  40. S.-F. Guo, F. Chen, Q. Liu, M. Xue, J.-J. Chen, J.-H. Cao, T.-W. Mao, M. K. Tey, and L. You, Faster state preparation across quantum phase transition assisted by reinforcement learning, Phys. Rev. Lett. 126, 060401 (2021).
  41. H. Xu, T. Xiao, J. Huang, M. He, J. Fan, and G. Zeng, Toward Heisenberg limit without critical slowing down via quantum reinforcement learning, Phys. Rev. Lett. 134, 120803 (2025).
  42. A. Melkani, C. Gneiting, and F. Nori, Eigenstate extraction with neural-network tomography, Phys. Rev. A 102, 022412 (2020).
  43. F. Chen, J.-J. Chen, L.-N. Wu, Y.-C. Liu, and L. You, Extreme spin squeezing from deep reinforcement learning, Phys. Rev. A 100, 041801(R) (2019).
  44. J.-H. Cao, F. Chen, Q. Liu, T.-W. Mao, W.-X. Xu, L.-N. Wu, and L. You, Detection of entangled states supported by reinforcement learning, Phys. Rev. Lett. 131, 073201 (2023).
  45. Y. Zhu, T. Xiao, G. Zeng, G. Chiribella, and Y.-D. Wu, Controlling unknown quantum states via data-driven state representations, npj Quantum Inf. (2026), doi:10.1038/s41534-026-01269-0.
  46. V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios, and M. H. Devoret, Model-free quantum control with reinforcement learning, Phys. Rev. X 12, 011059 (2022).
  47. Y. Yu, Y. Qiu, X. Zhao, Y.-H. Chen, and Y. Xia, Pulse parameterization and seeded optimization for fast longitudinal qubit readout, Phys. Rev. Appl. (2026), doi:10.1103/1dg9-b4vw.
  48. K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, Deep reinforcement learning: A brief survey, IEEE Signal Process. Mag. 34, 26 (2017).
  49. J. Yao, L. Lin, and M. Bukov, Reinforcement learning for many-body ground-state preparation inspired by counterdiabatic driving, Phys. Rev. X 11, 031070 (2021).
  50. T.-N. Xu, Y. Ding, J. D. Martín-Guerrero, and X. Chen, Robust two-qubit gate with reinforcement learning and dropout, Phys. Rev. A 110, 032614 (2024).
  51. V. Sivak, M. Newman, and P. Klimov, Optimization of decoder priors for accurate quantum error correction, Phys. Rev. Lett. 133, 150603 (2024).
  52. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
  53. H. Walther, B. T. H. Varcoe, B.-G. Englert, and T. Becker, Cavity quantum electrodynamics, Rep. Prog. Phys. 69, 1325 (2006).
  54. J. Johansson, P. Nation, and F. Nori, QuTiP: An open-source Python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 183, 1760 (2012).
  55. J. Johansson, P. Nation, and F. Nori, QuTiP 2: A Python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 184, 1234 (2013).
  56. T. Niemczyk, F. Deppe, H. Huebl, E. P. Menzel, F. Hocke, M. J. Schwarz, J. J. Garcia-Ripoll, D. Zueco, T. Hümmer, E. Solano, A. Marx, and R. Gross, Circuit quantum electrodynamics in the ultrastrong-coupling regime, Nat. Phys. 6, 772 (2010).
  57. X. Gu, A. F. Kockum, A. Miranowicz, Y.-x. Liu, and F. Nori, Microwave photonics with superconducting quantum circuits, Phys. Rep. 718-719, 1 (2017).
  58. M.-A. Lemonde, N. Didier, and A. A. Clerk, Enhanced nonlinear interactions in quantum optomechanics via mechanical amplification, Nat. Commun. 7, 11338 (2016).
  59. P. D. Nation, J. R. Johansson, M. P. Blencowe, and F. Nori, Colloquium: Stimulating uncertainty: Amplifying the quantum vacuum with superconducting circuits, Rev. Mod. Phys. 84, 1 (2012).
  60. D. Ballester, G. Romero, J. J. García-Ripoll, F. Deppe, and E. Solano, Quantum simulation of the ultrastrong-coupling dynamics in circuit quantum electrodynamics, Phys. Rev. X 2, 021007 (2012).
  61. M. Mirrahimi, Z. Leghtas, V. V. Albert, S. Touzard, R. J. Schoelkopf, L. Jiang, and M. H. Devoret, Dynamically protected cat-qubits: A new paradigm for universal quantum computation, New J. Phys. 16, 045014 (2014).
  62. J.-H. Lin, K.-X. Yan, J. Song, Y.-H. Chen, and Y. Xia, Approaching the critical point through an exact ground-state evolution, Phys. Rev. Res. 8, 013275 (2026).
  63. Y.-H. Chen, R. Stassi, W. Qin, A. Miranowicz, and F. Nori, Fault-tolerant multiqubit geometric entangling gates using photonic cat-state qubits, Phys. Rev. Appl. 18, 024076 (2022).
  64. M. M. Rams, P. Sierant, O. Dutta, P. Horodecki, and J. Zakrzewski, At the limits of criticality-based quantum metrology: Apparent super-Heisenberg scaling revisited, Phys. Rev. X 8, 021022 (2018).
  65. T. Lindeberg, Discrete approximations of Gaussian smoothing and Gaussian derivatives, J. Math. Imaging Vision 66, 759 (2024).
  66. K. Gietka, F. Metz, T. Keller, and J. Li, Adiabatic critical quantum metrology cannot reach the Heisenberg limit even when shortcuts to adiabaticity are applied, Quantum 5, 489 (2021).
  67. N. Lambert, E. Giguère, P. Menczel, B. Li, P. Hopf, G. Suárez, M. Galí, J. Lishman, R. Gadhvi, R. Agarwal, A. Galicia, N. Shammah, P. Nation, J. R. Johansson, S. Ahmed, S. Cross, A. Pitchford, and F. Nori, QuTiP 5: The quantum toolbox in Python, Phys. Rep. 1153, 1 (2026).
  68. P.-B. Zhou, Numerical Analysis of Electromagnetic Fields (Springer, Berlin, 1993), pp. 63–94.

Outline

Information

Phys. Rev. Research 8, 033063– Published 15 July, 2026
  • Received 24 September 2025
  • Revised 14 April 2026
  • Accepted 24 June 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation