- Open Access
Molecular quantum control algorithm design by reinforcement learning
Phys. Rev. Research 8, 033103 – Published 24 July, 2026
DOI: https://doi.org/10.1103/t159-pzlx
Abstract
Precision measurements of molecules offer an unparalleled paradigm to probe physics beyond the standard model. The rich internal structure within these molecules makes them exquisite sensors for detecting fundamental symmetry violations, local position invariance, and dark matter. While trapping and control of diatomic and a few very simple polyatomic molecules have been experimentally demonstrated, leveraging the complex rovibrational structure of more general polyatomics demands the development of robust and efficient quantum control algorithms. In this study, we present reinforcement-learning quantum-logic spectroscopy (RL-QLS), a general, reinforcement-learning-designed quantum-logic approach to prepare molecular ions in single, pure quantum states. The reinforcement-learning agent optimizes the pulse sequence, each followed by a projective measurement, and probabilistically manipulates the collapse of the quantum system to a single state. The performance of the RL-QLS control algorithm is numerically demonstrated for the polyatomic molecule with 130 thermally populated eigenstates and degenerate transitions within inversion doublets, as well as for under the disturbance of environmental thermal radiation. We expect that the developed theoretical framework can be readily implemented for quantum control of polyatomic molecular ions with densely populated structures, thereby facilitating experimental tests of fundamental theories.
Physics Subject Headings (PhySH)
Article Text
Supplemental Material
References (49)
- M. S. Safronova, D. Budker, D. DeMille, D. F. J. Kimball, A. Derevianko, and C. W. Clark, Search for new physics with atoms and molecules, Rev. Mod. Phys. 90, 025008 (2018).
- D. DeMille, N. R. Hutzler, A. M. Rey, and T. Zelevinsky, Quantum sensing and metrology for fundamental physics with molecules, Nat. Phys. 20, 741 (2024).
- M. Kozlov and S. Levshakov, Sensitivity of the inversion–rotational spectrum to changes in the electron-to-proton mass ratio, Astrophys. J. 726, 65 (2011).
- V. Letokhov, On difference of energy levels of left and right molecules due to weak interactions, Phys. Lett. A 53, 275 (1975).
- M. Quack, G. Seyfang, and G. Wichmann, Perspectives on parity violation in chiral molecules: Theory, spectroscopic experiment and biomolecular homochirality, Chem. Sci. 13, 10598 (2022).
- A. Landau, E. Eduardus, D. Behar, E. R. Wallach, L. F. Pašteka, S. Faraji, A. Borschevsky, and Y. Shagam, Chiral molecule candidates for trapped ion spectroscopy by ab initio calculations: From state preparation to parity violation, J. Chem. Phys. 159, 114307 (2023).
- D. Mitra, K. H. Leung, and T. Zelevinsky, Quantum control of molecules for fundamental physics, Phys. Rev. A 105, 040101 (2022).
- D. Patterson, Method for preparation and readout of polyatomic molecules in single quantum states, Phys. Rev. A 97, 033403 (2018).
- E. R. Hudson, Sympathetic cooling of molecular ions with ultracold atoms, EPJ Tech. Instrum. 3, 8 (2016).
- D. J. McCarron, M. H. Steinecker, Y. Zhu, and D. DeMille, Magnetic trapping of an ultracold gas of polar molecules, Phys. Rev. Lett. 121, 013202 (2018).
- S. Ospelkaus, K.-K. Ni, G. Quéméner, B. Neyenhuis, D. Wang, M. H. G. de Miranda, J. L. Bohn, J. Ye, and D. S. Jin, Controlling the hyperfine state of rovibronic ground-state polar molecules, Phys. Rev. Lett. 104, 030402 (2010).
- B. L. Augenbraun, J. M. Doyle, T. Zelevinsky, and I. Kozyryev, Molecular asymmetry and optical cycling: Laser cooling asymmetric top molecules, Phys. Rev. X 10, 031022 (2020).
- Y. Zeng, A. Jadbabaie, A. N. Patel, P. Yu, T. C. Steimle, and N. R. Hutzler, Optical cycling in polyatomic molecules with complex hyperfine structure, Phys. Rev. A 108, 012813 (2023).
- C. E. Dickerson, A. N. Alexandrova, P. Narang, and J. P. Philbin, Single molecule superradiance for optical cycling, arXiv:2310.01534.
- P. O. Schmidt, T. Rosenband, C. Langer, W. M. Itano, J. C. Bergquist, and D. J. Wineland, Spectroscopy using quantum logic, Science 309, 749 (2005).
- D. Leibfried, Quantum state preparation and control of single molecular ions, New J. Phys. 14, 023029 (2012).
- S. Ding and D. Matsukevich, Quantum logic for the control and manipulation of molecular ions using a frequency comb, New J. Phys. 14, 023028 (2012).
- C.-w. Chou, C. Kurz, D. B. Hume, P. N. Plessow, D. R. Leibrandt, and D. Leibfried, Preparation and coherent manipulation of pure quantum states of a single molecular ion, Nature (London) 545, 203 (2017).
- Y. Lin, D. R. Leibrandt, D. Leibfried, and C.-W. Chou, Quantum entanglement between an atom and a molecule, Nature (London) 581, 273 (2020).
- C.-w. Chou, A. L. Collopy, C. Kurz, Y. Lin, M. E. Harding, P. N. Plessow, T. Fortier, S. Diddams, D. Leibfried, and D. R. Leibrandt, Frequency-comb spectroscopy on pure quantum states of a single molecular ion, Science 367, 1458 (2020).
- Y. Liu, J. Schmidt, Z. Liu, D. R. Leibrandt, D. Leibfried, and C.-W. Chou, Quantum state tracking and control of a single molecular ion in a thermal environment, Science 385, 790 (2024).
- D. Holzapfel, F. Schmid, N. Schwegler, O. Stadler, M. Stadler, A. Ferk, J. P. Home, and D. Kienzler, Quantum control of a single molecular ion, Phys. Rev. X 15, 031009 (2025).
- M. Sinhal, Z. Meir, K. Najafian, G. Hegi, and S. Willitsch, Quantum-nondemolition state detection and spectroscopy of single trapped molecules, Science 367, 1213 (2020).
- F. Wolf, Y. Wan, J. C. Heip, F. Gebert, C. Shi, and P. O. Schmidt, Non-destructive state detection for quantum logic spectroscopy of molecular ions, Nature (London) 530, 457 (2016).
- X.-M. Zhang, Z. Wei, R. Asad, X.-C. Yang, and X. Wang, When does reinforcement learning stand out in quantum control? A comparative study on state preparation, npj Quantum Inf. 5, 85 (2019).
- Z. An, H.-J. Song, Q.-K. He, and D. L. Zhou, Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning, Phys. Rev. A 103, 012404 (2021).
- J. Mackeprang, D. B. R. Dasari, and J. Wrachtrup, A reinforcement learning approach for quantum state engineering, Quantum Mach. Intell. 2, 5 (2020).
- I. Paparelle, L. Moro, and E. Prati, Digitally stimulated Raman passage by deep reinforcement learning, Phys. Lett. A 384, 126266 (2020).
- M. Y. Niu, S. Boixo, V. N. Smelyanskiy, and H. Neven, Universal quantum control through deep reinforcement learning, npj Quantum Inf. 5, 33 (2019).
- F. Preti, M. Schilling, S. Jerbi, L. M. Trenkwalder, H. P. Nautrup, F. Motzoi, and H. J. Briegel, Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning, Quantum 8, 1343 (2024).
- See Supplemental Material at http://link.aps.org/supplemental/10.1103/t159-pzlx for Sec. SA: State evolution in the preparation process; Sec. SB: Time evolution with the adiabatically eliminated Hamiltonian; Secs. SC and SD: Computational details, and Figs. S1–S12, which also include Refs. [42, 43, 44, 45, 46, 47, 48, 49].
- R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 2018).
- V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, Playing Atari with deep reinforcement learning, arXiv:1312.5602.
- V. Mnih et al., Human-level control through deep reinforcement learning, Nature (London) 518, 529 (2015).
- C. J. Watkins, Learning from delayed rewards, Ph.D. thesis, King’s College, Cambridge, UK, 1989.
- A. Paszke et al., PyTorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems 32 (NeurIPS, 2019), pp. 8024–8035.
- D. Chaffee, B. Margulis, A. Sheffield, J. Schmidt, A. Reisenfeld, D. R. Leibrandt, D. Leibfried, and C.-W. Chou, High-fidelity quantum state control of a polar molecular ion in a cryogenic environment, Phys. Rev. Lett. 135, 240801 (2025).
- A. Wu et al., Prospects of local position invariance measurement with quantum logic spectroscopy of a hydronium ion, (unpublished).
- J. Barry, D. T. Barry, and S. Aaronson, Quantum partially observable Markov decision processes, Phys. Rev. A 90, 032311 (2014).
- M. L. Littman, A. R. Cassandra, and L. P. Kaelbling, Learning policies for partially observable environments: Scaling up, in Machine Learning Proceedings (Morgan Kaufmann, San Francisco, CA, 1995), pp. 362–370.
- M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, 2010).
- J. R. Johansson, P. D. Nation, and F. Nori, QuTiP: An open-source Python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 183, 1760 (2012).
- J. Johansson, P. Nation, and F. Nori, QuTiP 2: A Python framework for the dynamics of open quantum systems, Comput. Phys. Commun. 184, 1234 (2013).
- A. L. Collopy, J. Schmidt, D. Leibfried, D. R. Leibrandt, and C.-W. Chou, Effects of an oscillating electric field on and dipole moment measurement of a single molecular ion, Phys. Rev. Lett. 130, 223201 (2023).
- C. H. Townes and A. L. Schawlow, Microwave Spectroscopy (McGraw-Hill, New York, 1955).
- A. Corney, Atomic and Laser Spectroscopy (Clarendon Press, Oxford, 1977).
- P. Virtanen, et al., SciPy 1.0: Fundamental algorithms for scientific computing in Python, Nat. Methods 17, 261 (2020).
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
- J. Huerta-Cepas, F. Serra, and P. Bork, ETE 3: Reconstruction, analysis, and visualization of phylogenomic data, Mol. Biol. Evol. 33, 1635 (2016).