- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Exploring fixed points and eigenstates of quantum systems with reinforcement learning

María Laura Olivera-Atencio1,*, Jesús Casado-Pascual1,2,†, and Denis Lacroix3,‡

  • *Contact author: molivera1@us.es
  • †Contact author: jcasado@us.es
  • ‡Contact author: lacroix@ijclab.in2p3.fr
PDF

Phys. Rev. Research 8, 033087 – Published 22 July, 2026

DOI: https://doi.org/10.1103/vl66-7flb

Abstract

We introduce a reinforcement-learning algorithm designed to identify the fixed points of a given quantum operation. The method iteratively constructs the unitary transformation that maps the computational basis onto the basis of fixed points through a reward-penalty scheme based on quantum measurements. In cases where the operation corresponds to a Hamiltonian evolution, this task reduces to determining the Hamiltonian eigenstates. The algorithm is first benchmarked on random Hamiltonians acting on two and three qubits and then applied to many-body systems of up to six qubits, including the transverse-field Ising model and the all-to-all pairing Hamiltonian. In both cases, the algorithm is demonstrated to perform successfully; in the pairing model, it can also reveal hidden symmetries that can be exploited to restrict learning to specific symmetry sectors. Finally, we discuss the possibility of postselecting high-fidelity states even when full convergence has not been reached.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (89)

  1. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, 10th anniversary ed. (Cambridge University Press, Cambridge, 2010).
  2. M. Schuld, I. Sinayskiy, and F. Petruccione, An introduction to quantum machine learning, Contemp. Phys. 56, 172 (2015).
  3. G. Torlai, G. Mazzola, J. Carrasquilla, M. Troyer, R. Melko, and G. Carleo, Neural-network quantum state tomography, Nat. Phys. 14, 447 (2018).
  4. J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, Quantum machine learning, Nature (London) 549, 195 (2017).
  5. G. Liu, M. Chen, Y.-X. Liu, D. Layden, and P. Cappellaro, Repetitive readout enhanced by machine learning, Mach. Learn.: Sci. Technol. 1, 015003 (2020).
  6. R. Heese, T. Gerlach, S. Mücke, S. Müller, M. Jakobs, and N. Piatkowski, Explaining quantum circuits with Shapley values: Towards explainable quantum machine learning, Quantum Mach. Intell. 7, 27 (2025).
  7. T. Haug, C. N. Self, and M. S. Kim, Quantum machine learning of large datasets using randomized measurements, Mach. Learn.: Sci. Technol. 4, 015005 (2023).
  8. E. Perrier, D. Tao, and C. Ferrie, Quantum geometric machine learning for quantum circuits and control, New J. Phys. 22, 103056 (2020).
  9. F. A. Cárdenas-López, M. Sanz, J. C. Retamal, and E. Solano, Enhanced quantum synchronization via quantum machine learning, Adv. Quantum Technol. 2, 1800076 (2019).
  10. S. Saini, P. Khosla, M. Kaur, and G. Singh, Quantum driven machine learning, Int. J. Theor. Phys. 59, 4013 (2020).
  11. N. Innan, O. I. Siddiqui, S. Arora, T. Ghosh, Y. P. Koçak, D. Paragas, A. A. O. Galib, M. A.-Z. Khan, and M. Bennai, Quantum state tomography using quantum machine learning, Quantum Mach. Intell. 6, 28 (2024).
  12. R. Sutton and A. Barto, Reinforcement Learning: An Introduction., Adaptive Computation and Machine Learning, 2nd ed. (MIT Press, Cambridge, 2018).
  13. S. Shrapnel, F. Costa, and G. Milburn, Quantum Markovianity as a supervised learning task, Int. J. Quantum Inf. 16, 1840010 (2018).
  14. I. Agresti, N. Viggianiello, F. Flamini, N. Spagnolo, A. Crespi, R. Osellame, N. Wiebe, and F. Sciarrino, Pattern recognition techniques for boson sampling validation, Phys. Rev. X 9, 011013 (2019).
  15. A. Youssry, R. J. Chapman, A. Peruzzo, C. Ferrie, and M. Tomamichel, Modeling and control of a reconfigurable photonic circuit using deep learning, Quantum Sci. Technol. 5, 025001 (2020).
  16. I. A. Luchnikov, S. V. Vintskevich, D. A. Grigoriev, and S. N. Filippov, Machine learning non-Markovian quantum dynamics, Phys. Rev. Lett. 124, 140502 (2020).
  17. A. A. Melnikov, L. E. Fedichkin, R.-K. Lee, and A. Alodjants, Machine learning transfer efficiencies for noisy quantum walks, Adv. Quantum Technol. 3, 1900115 (2020).
  18. B. Lucian, B. Robert, S. B. De, and E. Damien, Reinforcement Learning and Dynamic Programming Using Function Approximators, 1st ed. (CRC Press Inc., Boca Raton, FL, 2010).
  19. F. Agostinelli, G. Hocquet, S. Singh, and P. Baldi, From reinforcement learning to deep reinforcement learning: An overview, in Braverman Readings in Machine Learning. Key Ideas from Inception to Current State: International Conference Commemorating the 40th Anniversary of Emmanuil Braverman's Decease, Boston, MA, USA, April 28–30, 2017, Invited Talks, edited by L. Rozonoer, B. Mirkin, and I. Muchnik (Springer International Publishing, Cham, 2018), pp. 298–328.
  20. D. Dong, C. Chen, H. Li, and T.-J. Tarn, Quantum reinforcement learning, IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 38, 1207 (2008).
  21. G. D. Paparo, V. Dunjko, A. Makmal, M. A. Martin-Delgado, and H. J. Briegel, Quantum speedup for active learning agents, Phys. Rev. X 4, 031002 (2014).
  22. V. Dunjko, J. M. Taylor, and H. J. Briegel, Quantum-enhanced machine learning, Phys. Rev. Lett. 117, 130501 (2016).
  23. M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, Reinforcement learning in different phases of quantum control, Phys. Rev. X 8, 031086 (2018).
  24. M. Bukov, Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator, Phys. Rev. B 98, 224305 (2018).
  25. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement learning with neural networks for quantum feedback, Phys. Rev. X 8, 031084 (2018).
  26. J. Kaldari, S. Tariq, S. Al-Kuwari, S. Y.-C. Chen, S. Chatzinotas, and H. Shin, Quantum reinforcement learning: Recent advances and future directions, arXiv:2510.14595.
  27. C. Jiang, Y. Pan, Z.-G. Wu, Q. Gao, and D. Dong, Robust optimization for quantum reinforcement learning control using partial observations, Phys. Rev. A 105, 062443 (2022).
  28. E. Cherrat, I. Kerenidis, and A. Prakash, Quantum reinforcement learning via policy iteration, Quantum Mach. Intell. 5, 30 (2023).
  29. Y. Liu, C. Xu, and S. Jin, Reinforcement learning for continuous control: A quantum normalized advantage function approach, in Proceedings of the 2023 IEEE International Conference on Quantum Software (QSW) (IEEE, Piscataway, NJ, 2023), pp. 83–87.
  30. G. Franceschetto and A. Ricou, Demonstration of quantum projective simulation on a single-photon-based quantum computer, Phys. Rev. A 110, 062613 (2024).
  31. M. L. Olivera-Atencio, L. Lamata, M. Morillo, and J. Casado-Pascual, Quantum reinforcement learning in the presence of thermal dissipation, Phys. Rev. E 108, 014128 (2023).
  32. M. L. Olivera-Atencio, L. Lamata, and J. Casado-Pascual, Benefits of open quantum systems for quantum machine learning, Adv. Quantum Technol. 8, 2300247 (2023).
  33. M. L. Olivera-Atencio, L. Lamata, and J. Casado-Pascual, Impact of amplitude and phase damping noise on quantum reinforcement learning: Challenges and opportunities, Eur. Phys. J. Spec. Top. (2025).
  34. F. Albarrán-Arriagada, J. C. Retamal, E. Solano, and L. Lamata, Reinforcement learning for semi-autonomous approximate quantum eigensolver, Mach. Learn.: Sci. Technol. 1, 015002 (2020).
  35. J. R. McClean, J. Romero, R. Babbush, and A. Aspuru-Guzik, The theory of variational hybrid quantum-classical algorithms, New J. Phys. 18, 023023 (2016).
  36. S. McArdle, S. Endo, A. Aspuru-Guzik, S. C. Benjamin, and X. Yuan, Quantum computational chemistry, Rev. Mod. Phys. 92, 015003 (2020).
  37. M. Motta and J. E. Rice, Emerging quantum computing algorithms for quantum chemistry, WIREs Comput. Mol. Sci. 12, e1580 (2022).
  38. M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin, S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan, L. Cincio, and P. J. Coles, Variational quantum algorithms, Nat. Rev. Phys. 3, 625 (2021).
  39. J. Tilly, H. Chen, S. Cao, D. Picozzi, K. Setia, Y. Li, E. Grant, L. Wossnig, I. Rungger, G. H. Booth, and J. Tennyson, The variational quantum eigensolver: A review of methods and best practices, Phys. Rep. 986, 1 (2022).
  40. M. Larocca, S. Thanasilp, S. Wang, K. Sharma, J. Biamonte, P. J. Coles, L. Cincio, J. R. McClean, Z. Holmes, and M. Cerezo, Barren plateaus in variational quantum computing, Nat. Rev. Phys. 7, 174 (2025).
  41. J. R. McClean, M. E. Kimchi-Schwartz, J. Carter, and W. A. de Jong, Hybrid quantum-classical hierarchy for mitigation of decoherence and determination of excited states, Phys. Rev. A 95, 042308 (2017).
  42. H. R. Grimsley, S. E. Economou, E. Barnes, and N. J. Mayhall, An adaptive variational algorithm for exact molecular simulations on a quantum computer, Nat. Commun. 10, 3007 (2019).
  43. K. M. Nakanishi, K. Mitarai, and K. Fujii, Subspace-search variational quantum eigensolver for excited states, Phys. Rev. Res. 1, 033062 (2019).
  44. R. M. Parrish, E. G. Hohenstein, P. L. McMahon, and T. J. Martínez, Quantum computation of electronic transitions using a variational quantum eigensolver, Phys. Rev. Lett. 122, 230401 (2019).
  45. O. Higgott, D. Wang, and S. Brierley, Variational quantum computation of excited states, Quantum 3, 156 (2019).
  46. T. Jones, S. Endo, S. McArdle, X. Yuan, and S. C. Benjamin, Variational quantum algorithms for discovering Hamiltonian spectra, Phys. Rev. A 99, 062304 (2019).
  47. M. Motta, C. Sun, A. T. K. Tan, M. J. O’Rourke, E. Ye, A. J. Minnich, F. G. S. L. Brandão, and G. K.-L. Chan, Determining eigenstates and thermal states on a quantum computer using quantum imaginary time evolution, Nat. Phys. 16, 205 (2020).
  48. W. J. Huggins, J. Lee, U. Baek, B. O’Gorman, and K. B. Whaley, A non-orthogonal variational quantum eigensolver, New J. Phys. 22, 073009 (2020).
  49. N. H. Stair, R. Huang, and F. A. Evangelista, A multireference quantum Krylov algorithm for strongly correlated electrons, J. Chem. Theory Comput. 16, 2236 (2020).
  50. K. Seki and S. Yunoki, Quantum power method by a superposition of time-evolved states, PRX Quantum 2, 010333 (2021).
  51. Y. S. Yordanov, V. Armaos, C. H. W. Barnes, and D. R. M. Arvidsson-Shukur, Qubit-excitation-based adaptive variational quantum eigensolver, Commun. Phys. 4, 228 (2021).
  52. H. L. Tang, V. O. Shkolnikov, G. S. Barron, H. R. Grimsley, N. J. Mayhall, E. Barnes, and S. E. Economou, Qubit-ADAPT-VQE: An adaptive algorithm for constructing hardware-efficient ansätze on a quantum processor, PRX Quantum 2, 020310 (2021).
  53. F. Zhang, N. Gomes, Y. Yao, P. P. Orth, and T. Iadecola, Adaptive variational quantum eigensolvers for highly excited states, Phys. Rev. B 104, 075159 (2021).
  54. Y. S. Yordanov, C. H. W. Barnes, and D. R. M. Arvidsson-Shukur, Molecular-excited-state calculations with the qubit-excitation-based adaptive variational quantum eigensolver protocol, Phys. Rev. A 106, 032434 (2022).
  55. C. L. Cortes and S. K. Gray, Quantum Krylov subspace algorithms for ground- and excited-state energy estimation, Phys. Rev. A 105, 022417 (2022).
  56. E. N. Epperly, L. Lin, and Y. Nakatsukasa, A theory of quantum subspace diagonalization, SIAM J. Matrix Anal. Appl. 43, 1263 (2022).
  57. E. A. R. Guzman and D. Lacroix, Accessing ground-state and excited-state energies in a many-body system after symmetry restoration using quantum computers, Phys. Rev. C 105, 024324 (2022).
  58. M. Q. Hlatshwayo, Y. Zhang, H. Wibowo, R. LaRose, D. Lacroix, and E. Litvinova, Simulating excited states of the Lipkin model on a quantum computer, Phys. Rev. C 106, 024319 (2022).
  59. Y. Wang and D. A. Mazziotti, Electronic excited states from a variance-based contracted quantum eigensolver, Phys. Rev. A 108, 022814 (2023).
  60. Y. Shen, K. Klymko, J. Sud, D. B. Williams-Young, W. A. de Jong, and N. M. Tubman, Real-time Krylov theory for quantum computing algorithms, Quantum 7, 1066 (2023).
  61. M. Zheng, B. Peng, N. Wiebe, A. Li, X. Yang, and K. Kowalski, Quantum algorithms for generator coordinate methods, Phys. Rev. Res. 5, 023200 (2023).
  62. S. E. Smart, D. M. Welakuh, and P. Narang, Many-body excited states with a contracted quantum eigensolver, J. Chem. Theory Comput. 20, 3580 (2024).
  63. M. Motta, W. Kirby, I. Liepuoniute, K. J. Sung, J. Cohn, A. Mezzacapo, K. Klymko, N. Nguyen, N. Yoshioka, and J. E. Rice, Subspace methods for electronic structure simulations on quantum computers, Electron. Struct. 6, 013001 (2024).
  64. M. R. Hirsbrunner, J. W. Mullinax, Y. Shen, D. B. Williams-Young, K. Klymko, R. V. Beeumen, and N. M. Tubman, A circuit-generated quantum subspace algorithm for the variational quantum eigensolver, J. Chem. Phys. 161, 164103 (2024).
  65. M. Zheng, B. Peng, A. Li, X. Yang, and K. Kowalski, Unleashed from constrained optimization: Quantum computing for quantum chemistry employing generator coordinate inspired method, NPJ Quantum Inf. 10, 127 (2024).
  66. Y. O. Nakagawa, M. Kamoshita, W. Mizukami, S. Sudo, and Y. ya Ohnishi, ADAPT-QSCI: Adaptive construction of an input state for quantum-selected configuration interaction, J. Chem. Theory Comput. 20, 10817 (2024).
  67. L. Ding, C.-L. Hong, and C. Schilling, Ground and excited states from ensemble variational principles, Quantum 8, 1525 (2024).
  68. Y. Beaujeault-Taudière and D. Lacroix, Solving the Lipkin model using quantum computers with two qubits only with a hybrid quantum-classical technique based on the generator coordinate method, Phys. Rev. C 109, 024327 (2024).
  69. M. Q. Hlatshwayo, J. Novak, and E. Litvinova, Quantum benefit of the quantum equation of motion for the strongly coupled many-body problem, Phys. Rev. C 109, 014306 (2024).
  70. S. Barison, J. R. Moreno, and M. Motta, Quantum-centric computation of molecular excited states with extended sample-based quantum diagonalization, Quantum Sci. Technol. 10, 025034 (2025).
  71. D. Marti-Dafcik, H. G. A. Burton, and D. P. Tew, Spin coupling is all you need: Encoding strong electron correlation in molecules on quantum computers, Phys. Rev. Res. 7, 013191 (2025).
  72. H. R. Grimsley and F. A. Evangelista, Challenging excited states from adaptive quantum eigensolvers: Subspace expansions vs state-averaged strategies, Quantum Sci. Technol. 10, 025003 (2025).
  73. J. Zhang and D. Lacroix, Excited states from ADAPT-VQE convergence path in many-body problems: Application to nuclear pairing problem and H4 molecule dissociation, Phys. Lett. B 869, 139841 (2025).
  74. M. A. Caprio, P. Cejnar, and F. Iachello, Excited-state quantum phase transitions in many-body systems, Ann. Phys. (NY) 323, 1106 (2008).
  75. O. Bohigas, M. J. Giannoni, and C. Schmit, Characterization of chaotic quantum spectra and universality of level fluctuation laws, Phys. Rev. Lett. 52, 1 (1984).
  76. F. Haake, Quantum Signatures of Chaos, 3rd ed. (Springer, Berlin, Heidelberg, 2010).
  77. M. Srednicki, Chaos and quantum thermalization, Phys. Rev. E 50, 888 (1994).
  78. L. D’Alessio, Y. Kafri, A. Polkovnikov, and M. Rigol, From quantum chaos and eigenstate thermalization to statistical mechanics and thermodynamics, Adv. Phys. 65, 239 (2016).
  79. I. M. Georgescu, S. Ashhab, and F. Nori, Quantum simulation, Rev. Mod. Phys. 86, 153 (2014).
  80. M. Bukov, L. D’Alessio, and A. Polkovnikov, Universal high-frequency behavior of periodically driven systems: From dynamical stabilization to Floquet engineering, Adv. Phys. 64, 139 (2015).
  81. A. Eckardt, Colloquium: Atomic quantum gases in periodically driven optical lattices, Rev. Mod. Phys. 89, 011004 (2017).
  82. T. Lattimore and C. Szepesvári, Bandit Algorithms (Cambridge University Press, Cambridge, 2020).
  83. E. Fradking, Field Theories of Condensed Matter Physics (Cambridge University Press, Cambridge, 2013).
  84. B. Anselme Martin, T. Ayral, F. Jamet, M. J. Rančić, and P. Simon, Combining matrix product states and noisy quantum computers for quantum simulation, Phys. Rev. A 109, 062437 (2024).
  85. J. Dukelsky, S. Pittel, and G. Sierra, Colloquium: Exactly solvable Richardson-Gaudin models for many-body quantum systems, Rev. Mod. Phys. 76, 643 (2004).
  86. J. von Delft and D. Ralph, Spectroscopy of discrete energy levels in ultrasmall metallic grains, Phys. Rep. 345, 61 (2001).
  87. V. Zelevinsky and A. Volya, Nuclear pairing: New perspectives, Phys. At. Nucl. 66, 1781 (2003).
  88. D. M. Brink and R. A. Broglia, Nuclear Superfluidity Pairing in Finite Systems (Cambridge University Press, Cambridge, 2005).
  89. www.hqi.fr.

Outline

Information

Phys. Rev. Research 8, 033087– Published 22 July, 2026
  • Received 21 November 2025
  • Accepted 11 June 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation