- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Reinforcement learning for ion shuttling on trapped-ion quantum computers

Maximilian Schier1,*,†, Lea Richtmann2,*,‡, Christian Staufenbiel3, Tobias Schmale3,4, Daniel Borcherding3, Michèle Heurs2,5,6, and Bodo Rosenhahn1

  • *These authors contributed equally to this work.
  • †Contact author: schier@tnt.uni-hannover.de
  • ‡Contact author: lea.richtmann@aei.uni-hannover.de
PDF

Phys. Rev. Research 8, 033360 – Published 24 September, 2026

DOI: https://doi.org/10.1103/b7ck-8wh4

Abstract

Scalable trapped-ion quantum computing is commonly realized with modular chips that feature distinct zones with specific functionalities, such as storage, state preparation, and gate execution. To execute a quantum circuit, the ions must be transported between these zones. This process is called ion shuttling. To achieve reliable computation results, the shuttling process must be optimized. However, as the number of ions increases, this becomes a high-dimensional optimization problem where optimal solutions cannot be computed efficiently. We demonstrate, to the best of our knowledge, the first use of reinforcement learning (RL) for the optimization of ion shuttling. RL is well suited for such scenarios, as it enables learning a strategy through direct interaction with the problem. We show that our RL approach outperforms current state-of-the-art heuristic techniques, yielding a reduction in shuttling operations of up to 36.3%. Furthermore, we show that our method is easily applicable to various chip architectures. Our approach offers a versatile method to study shuttling efficiency during chip design and, therefore, a highly relevant tool for future, more complex architectures.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (51)

  1. C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, Trapped-ion quantum computing: Progress and challenges, Appl. Phys. Rev. 6, 021314 (2019).
  2. J. I. Cirac and P. Zoller, Quantum computations with cold trapped ions, Phys. Rev. Lett. 74, 4091 (1995).
  3. A. Sørensen and K. Mølmer, Quantum computation with ions in thermal motion, Phys. Rev. Lett. 82, 1971 (1999).
  4. G. Zarantonello, H. Hahn, J. Morgner, M. Schulte, A. Bautista-Salvador, R. F. Werner, K. Hammerer, and C. Ospelkaus, Robust and resource-efficient microwave near-field entangling 9Be+ gate, Phys. Rev. Lett. 123, 260503 (2019).
  5. D. Kielpinski, C. Monroe, and D. J. Wineland, Architecture for a large-scale ion-trap quantum computer, Nature (London) 417, 709 (2002).
  6. J. M. Pino, J. M. Dreiling, C. Figgatt, J. P. Gaebler, S. A. Moses, M. S. Allman, C. H. Baldwin, M. Foss-Feig, D. Hayes, K. Mayer, C. Ryan-Anderson, and B. Neyenhuis, Demonstration of the trapped-ion quantum CCD computer architecture, Nature (London) 592, 209 (2021).
  7. S. A. Moses, C. H. Baldwin, M. S. Allman, R. Ancona, L. Ascarrunz, C. Barnes, J. Bartolotta, B. Bjork, P. Blanchard, M. Bohn, et al., A race-track trapped-ion quantum processor, Phys. Rev. X 13, 041052 (2023).
  8. J. Durandau, J. Wagner, F. Mailhot, C.-A. Brunet, F. Schmidt-Kaler, U. Poschinger, and Y. Bérubé-Lauzière, Automated generation of shuttling sequences for a linear segmented ion trap quantum computer, Quantum 7, 1175 (2023).
  9. A. Ransford, M. S. Allman, J. Arkinstall, J. P. Campora III, S. F. Cooper, R. D. Delaney, J. M. Dreiling, B. Estey, C. Figgatt, A. Hall, et al., Helios: A 98-qubit trapped-ion quantum computer, arXiv:2511.05465.
  10. T. Schmale, B. Temesi, A. Baishya, N. Pulido-Mateo, L. Krinner, T. Dubielzig, C. Ospelkaus, H. Weimer, and D. Borcherding, Backend compiler phases for trapped-ion quantum computers, in 2022 IEEE International Conference on Quantum Software (QSW) (IEEE, Barcelona, Spain, 2022), pp. 32–37.
  11. A. A. Saki, R. O. Topaloglu, and S. Ghosh, Muzzle the shuttle: Efficient compilation for multi-trap trapped-ion quantum computers, in 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE) (IEEE, Antwerp, Belgium, 2022), pp. 322–327.
  12. P. Murali, D. M. Debroy, K. R. Brown, and M. Martonosi, Architecting noisy intermediate-scale trapped ion quantum computers, in 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) (IEEE, Valencia, Spain, 2020), pp. 529–542.
  13. X. Wu, C. Zhu, J. Wang, and X. Wang, MUSS-TI: Multi-level shuttle scheduling for large-scale entanglement module linked trapped-ion, in Proceedings of the 58th IEEE/ACM International Symposium on Microarchitecture (Association for Computing Machinery, New York, NY, USA, 2025), pp. 749–763.
  14. D. Schoenberger, S. Hillmich, M. Brandl, and R. Wille, Shuttling for scalable trapped-ion quantum computers, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 44, 2144 (2025).
  15. W. Dai, K. A. Brown, and T. G. Robertazzi, Advanced shuttle strategies for parallel QCCD architectures, IEEE Trans. Quantum Eng. 5, 1 (2024).
  16. Quantum Valley Lower Saxony e. V., 2026, accessed: 20 May 2026, https://qvls.de/en/.
  17. R. Wille, L. Berent, T. Forster, J. Kunasaikaran, K. Mato, T. Peham, N. Quetschlich, D. Rovara, A. Sander, L. Schmid, D. Schoenberger, Y. Stade, and L. Burgholzer, The MQT handbook: A summary of design automation tools and software for quantum computing, in IEEE International Conference on Quantum Software (QSW) (IEEE, Shenzhen, China, 2024), pp. 1–8.
  18. A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, Validating quantum computers using randomized model circuits, Phys. Rev. A 100, 032328 (2019).
  19. Quantum Valley Lower Saxony Q1–Forschung, 2026, accessed: 20 May 2026, www.qvls-q1.de/forschung.
  20. D. Schoenberger, S. Hillmich, M. Brandl, and R. Wille, Using Boolean satisfiability for exact shuttling in trapped-ion quantum computers, in 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC) (IEEE, Incheon, Republic of Korea, 2024), pp. 127–133.
  21. R. B. Blakestad, Transport of trapped-ion qubits within a scalable quantum processor, Ph.D. thesis, University of Colorado, 2010.
  22. D. Hucul, M. Yeo, S. Olmschenk, C. Monroe, W.and K. Hensinger J. Rabchuk, On the transport of atomic ions in linear and multidimensional ion trap arrays, Quantum Info. Comput. 8, 501 (2008).
  23. F. Ungerechts, R. Munoz, A. Hoffmann, J. Bätge, M. M. Billah, T. Meiners, B. Kaune, G. Zarantonello, and C. Ospelkaus, Designing a trapped-ion quantum processor based on near-field microwave quantum logic gates (unpublished).
  24. F. Ungerechts, J. Bätge, M. M. Billah, L. Krieger, R. Munoz, P. Nuschke, A. Hoffmann, G. Zarantonello, and C. Ospelkaus, CIRQLE: A comprehensive register-based trapped-ion quantum processor with near-field microwave control (unpublished).
  25. R. Bowler, J. Gaebler, Y. Lin, T. R. Tan, D. Hanneke, J. D. Jost, J. P. Home, D. Leibfried, and D. J. Wineland, Coherent diabatic ion transport and separation in a multizone trap array, Phys. Rev. Lett. 109, 080502 (2012).
  26. A. Walther, F. Ziesel, T. Ruster, S. T. Dawkins, K. Ott, M. Hettrich, K. Singer, F. Schmidt-Kaler, and U. Poschinger, Controlling fast transport of cold trapped ions, Phys. Rev. Lett. 109, 080501 (2012).
  27. X.-J. Lu, A. Ruschhaupt, and J. G. Muga, Fast shuttling of a particle under weak spring-constant noise of the moving trap, Phys. Rev. A 97, 053402 (2018).
  28. V. Kaushal, B. Lekitsch, A. Stahl, J. Hilder, D. Pijn, C. Schmiegelow, A. Bermudez, M. Müller, F. Schmidt-Kaler, and U. Poschinger, Shuttling-based trapped-ion quantum information processing, AVS Quantum Sci. 2, 014101 (2020).
  29. D. Schoenberger and R. Wille, Orchestrating multi-zone shuttling in trapped-ion quantum computers, in 2025 IEEE International Conference on Quantum Computing and Engineering (QCE) (IEEE, Albuquerque, NM, USA, 2025), Vol. 1, p. 1069.
  30. D. Schoenberger, J. Hilder, F. Schmidt-Kaler, and R. Wille, Shuttling for trapped-ion quantum computers with embedded processing zones, in 2025 IEEE International Conference on Quantum Software (QSW) (IEEE, Helsinki, Finland, 2025), pp. 123–129.
  31. T. Schmale, Hybrid quantum-classical computation—From infrastructure to algorithms, Institutionelles Repositorium der Leibniz Universität Hannover, Dissertation, 2026.
  32. M. Sipser, Introduction to the theory of computation, SIGACT News 27, 27 (1996).
  33. X. Zhang and G.-Y. Zhu, A literature review of reinforcement learning methods applied to job-shop scheduling problems, Comput. Oper. Res. 175, 106929 (2025).
  34. M. Nazari, A. Oroojlooy, L. Snyder, and M. Takác, Reinforcement learning for solving the vehicle routing problem, in Proceedings of the 32nd International Conference on Advances in Neural Information Processing Systems (NeurIPS 2018) (Neural Information Processing Systems Foundation, Inc., Montréal, Canada, 2018), Vol. 31.
  35. C. Cummins, B. Wasti, J. Guo, B. Cui, J. Ansel, S. Gomez, S. Jain, J. Liu, O. Teytaud, B. Steiner, et al., Compilergym: Robust, performant compiler optimization environments for AI research, in 2022 IEEE/ACM International Symposium on Code Generation and Optimization (CGO) (IEEE, Seoul, Republic of Korea, 2022), pp. 92–105.
  36. M. Bukov and F. Marquardt, Reinforcement learning for quantum technology, arXiv:2601.18953.
  37. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley & Sons, Hoboken, NJ, USA, 2014).
  38. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., edited by F. Bach (MIT Press, Cambridge, MA, USA, 2018), pp. 47–71.
  39. R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, in Proceedings of the 12th International Conference on Advances in Neural Information Processing Systems (NeurIPS 1999) (MIT Press, Cambridge, MA, USA, 1999), Vol. 12.
  40. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
  41. J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel, High-dimensional continuous control using generalized advantage estimation, in 4th International Conference on Learning Representations, ICLR 2016 (OpenReview.net, San Juan, Puerto Rico, 2016).
  42. R. Givan, T. Dean, and M. Greig, Equivalence notions and model minimization in Markov decision processes, Artif. Intell. 147, 163 (2003).
  43. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Proceedings of the 31st International Conference on Advances in Neural Information Processing Systems (NeurIPS 2017) (Curran Associates, Incorporated, Red Hook, NY, USA, 2017), Vol. 30.
  44. L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning, et al., Impala: Scalable distributed deep-RL with importance weighted actor-learner architectures, in International Conference on Machine Learning (PMLR, Stockholm, Sweden, 2018), pp. 1407–1416.
  45. H. Lee, D. Hwang, D. Kim, H. Kim, J. J. Tai, K. Subramanian, P. R. Wurman, J. Choo, P. Stone, and T. Seno, Simba: Simplicity bias for scaling up parameters in deep reinforcement learning, in 13th International Conference on Learning Representations, ICLR 2025 (International Conference on Learning Representations, Singapore, 2025), pp. 50050–50082.
  46. A. Y. Ng, D. Harada, and S. Russell, Policy invariance under reward transformations: Theory and application to reward shaping, in 16th International Conference on Machine Learning (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1999), Vol. 99, p. 278.
  47. See Supplemental Material at https://link.aps.org/supplemental/10.1103/b7ck-8wh4 for all results on MQT circuits and animations of some shuttling sequences.
  48. B. Rosenhahn and T. J. Osborne, Monte Carlo graph search for quantum circuit optimization, Phys. Rev. A 108, 062615 (2023).
  49. B. Rosenhahn, T. J. Osborne, and C. Hirche, Optimization driven quantum circuit reduction, New J. Phys. 27, 104509 (2025).
  50. M. Schier and L. Richtmann, RLIonS: Reinforcement learning for ion shuttling, GitHub, 2026, https://github.com/m-schier/rlions/tree/v1.0.
  51. L. De Moura and N. Bjørner, Z3: An efficient SMT solver, in International Conference on Tools and Algorithms for the Construction and Analysis of Systems (Springer, Berlin, Heidelberg, Germany, 2008), pp. 337– 340.

Outline

Information

Phys. Rev. Research 8, 033360– Published 24 September, 2026
  • Received 21 May 2026
  • Accepted 6 August 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation