- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Quantum Circuit Discovery for Fault-Tolerant Logical State Preparation with Reinforcement Learning

Remmy Zen1,*, Jan Olle1, Luis Colmenarez2,3, Matteo Puviani1, Markus Müller2,3, and Florian Marquardt1,4

  • *Contact author: remmy.zen@mpl.mpg.de
PDF

Phys. Rev. X 15, 041012 – Published 22 October, 2025

DOI: https://doi.org/10.1103/gqpr-dgz7

Abstract

The realization of large-scale quantum computers requires not only quantum error correction but also fault-tolerant (FT) operations to handle errors that propagate into harmful errors. Recently, flag-based protocols have been introduced that use ancillary qubits to flag harmful errors. However, there is no clear recipe for finding a FT quantum circuit with flag-based protocols, especially when we consider hardware constraints, such as the qubit connectivity and available gate set. This work presents a novel approach to automatically discover compact and hardware-adapted FT quantum circuits to make significant progress towards scalable FT quantum computing. We employ reinforcement learning (RL) as an enabling tool, leveraging a fast, parallelized stabilizer quantum circuit simulator and a nontrivial reward function specifically adapted to the problem. We show that, in the task of FT logical state preparation, RL discovers not only circuits with fewer gates and ancillary qubits than published results but also novel circuits without and with hardware constraints of up to distance-5 codes with 25 physical qubits, and they can be implemented directly in experiments. Furthermore, RL allows for straightforward exploration of different qubit connectivities and the use of transfer learning to accelerate the discovery. More generally, our work sets the framework towards the use of RL or other machine learning techniques for FT quantum circuit discovery with hardware constraints to make real progress towards the realization of large-scale quantum computers, addressing tasks beyond state preparation, including magic state preparation, logical gate synthesis, and syndrome measurement.

View figure in article

Physics Subject Headings (PhySH)

Popular Summary

Quantum computers have the potential to solve certain problems much faster than classical computers, but they are also very sensitive to errors. Quantum error correction (QEC) is needed to keep errors in check. Unfortunately, some errors can spread in ways that QEC cannot fix. To handle this, fault-tolerant protocols use special “flag qubits”—quantum bits that signal when dangerous errors appear. Until now, most of these protocols have been designed by hand, often without considering the limits of real quantum hardware. In our work, we tackle this problem by using reinforcement learning, a branch of artificial intelligence (AI), to automatically design fault-tolerant quantum circuits.

Reinforcement learning has already shown great success in mastering difficult games and tasks by finding strategies beyond those developed by humans. We apply reinforcement learning to the key challenge of preparing logical quantum states in a fault-tolerant way, which is the first step of QEC. Our approach allows the AI to explore many possible circuit designs and learn which ones are most efficient while respecting hardware constraints. In doing so, the method discovers circuits that require fewer gates and flag qubits than the best human-designed circuits, as well as entirely new circuit structures.

While we focus here on logical state preparation, this is just one part of the full QEC framework. By extending our method, we can aim to automate other essential building blocks of error correction and fault tolerance, thus accelerating the path toward building efficient, large-scale quantum computers.

Article Text

References (145)

  1. J. Preskill, Quantum computing in the NISQ era and beyond, Quantum 2, 79 (2018).
  2. D. Gottesman, An introduction to quantum error correction and fault-tolerant quantum computation, in Proceedings of Symposia in Applied Mathematics, edited by S. Lomonaco (American Mathematical Society, Providence, Rhode Island, 2010), Vol. 68, pp. 13–58.
  3. L. Postler, S. Heußen, I. Pogorelov, M. Rispler, T. Feldker, M. Meth, C. D. Marciniak, R. Stricker, M. Ringbauer, R. Blatt, P. Schindler, M. Müller, and T. Monz, Demonstration of fault-tolerant universal quantum gate operations, Nature (London) 605, 675 (2022).
  4. I. Cong, H. Levine, A. Keesling, D. Bluvstein, S.-T. Wang, and M. D. Lukin, Hardware-efficient, fault-tolerant quantum computation with Rydberg atoms, Phys. Rev. X 12, 021049 (2022).
  5. S. Krinner, N. Lacroix, A. Remm, A. Di Paolo, E. Genois, C. Leroux, C. Hellings, S. Lazar, F. Swiadek, J. Herrmann, G. J. Norris, C. K. Andersen, M. Müller, A. Blais, C. Eichler, and A. Wallraff, Realizing repeated quantum error correction in a distance-three surface code, Nature (London) 605, 669 (2022).
  6. C. Ryan-Anderson et al., Implementing fault-tolerant entangling gates on the five-qubit code and the color code, arXiv:2208.01863.
  7. M. H. Abobeih, Y. Wang, J. Randall, S. J. H. Loenen, C. E. Bradley, M. Markham, D. J. Twitchen, B. M. Terhal, and T. H. Taminiau, Fault-tolerant operation of a logical qubit in a diamond quantum processor, Nature (London) 606, 884 (2022).
  8. Y. Zhao et al., Realization of an error-correcting surface code with superconducting qubits, Phys. Rev. Lett. 129, 030501 (2022).
  9. Y. Wang, S. Simsek, T. M. Gatterman, J. A. Gerber, K. Gilmore, D. Gresh, N. Hewitt, C. V. Horst, M. Matheny, T. Mengle, B. Neyenhuis, and B. Criger, Fault-tolerant one-bit addition with the smallest interesting color code, Sci. Adv. 10, eado9024 (2024).
  10. D. Bluvstein et al., Logical quantum processor based on reconfigurable atom arrays, Nature (London) 626, 58 (2024).
  11. K. Mayer et al., Benchmarking logical three-qubit quantum Fourier transform encoded in the Steane code on a trapped-ion quantum computer, arXiv:2404.08616.
  12. V. V. Sivak, A. Eickbusch, B. Royer, S. Singh, I. Tsioutsios, S. Ganjam, A. Miano, B. L. Brock, A. Z. Ding, L. Frunzio, S. M. Girvin, R. J. Schoelkopf, and M. H. Devoret, Real-time quantum error correction beyond break-even, Nature (London) 616, 50 (2023).
  13. R. Acharya, I. Aleiner, R. Allen et al., Suppressing quantum errors by scaling a surface code logical qubit, Nature (London) 614, 676 (2023).
  14. M. P. da Silva et al., Demonstration of logical qubits and repeated error correction with better-than-physical error rates, arXiv:2404.02280.
  15. D. Aharonov and M. Ben-Or, Fault-tolerant quantum computation with constant error rate, SIAM J. Comput. 38, 1207 (2008).
  16. E. Knill, R. Laflamme, and W. H. Zurek, Resilient quantum computation: Error models and thresholds, Proc. R. Soc. A 454, 365 (1998).
  17. A. Kitaev, Fault-tolerant quantum computation by anyons, Ann. Phys. (Amsterdam) 303, 2 (2003).
  18. E. T. Campbell, B. M. Terhal, and C. Vuillot, Roads towards fault-tolerant universal quantum computation, Nature (London) 549, 172 (2017).
  19. I. K. Sohn and J. Heo, An introduction to fault-tolerant quantum computation and its overhead reduction schemes, in 2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN) (IEEE, Prague, Czech Republic, 2018), pp. 44–46.
  20. R. Chao and B. W. Reichardt, Flag fault-tolerant error correction for any stabilizer code, PRX Quantum 1, 010302 (2020).
  21. S. Heußen, D. F. Locher, and M. Müller, Measurement-free fault-tolerant quantum error correction in near-term devices, PRX Quantum 5, 010333 (2024).
  22. J. Preskill, Reliable quantum computers, Proc. R. Soc. A 454, 385 (1998).
  23. P. Shor, Fault-tolerant quantum computation, in Proceedings of 37th Conference on Foundations of Computer Science (1996), pp. 56–65, 10.1109/SFCS.1996.548464.
  24. A. M. Steane, Active stabilization, quantum computation, and quantum state synthesis, Phys. Rev. Lett. 78, 2252 (1997).
  25. S. Huang, K. R. Brown, and M. Cetina, Comparing Shor and Steane error correction using the Bacon-Shor code, Sci. Adv. 10, eadp2008 (2024).
  26. L. Postler, F. Butt, I. Pogorelov, C. D. Marciniak, S. Heußen, R. Blatt, P. Schindler, M. Rispler, M. Müller, and T. Monz, Demonstration of fault-tolerant steane quantum error correction, PRX Quantum 5, 030326 (2024).
  27. T. J. Yoder and I. H. Kim, The surface code with a twist, Quantum 1, 2 (2017).
  28. C. Chamberland and M. E. Beverland, Flag fault-tolerant error correction with arbitrary distance codes, Quantum 2, 53 (2018).
  29. R. Chao and B. W. Reichardt, Quantum error correction with only two extra qubits, Phys. Rev. Lett. 121, 050502 (2018).
  30. C. Chamberland and K. Noh, Very low overhead fault-tolerant magic state preparation using redundant ancilla encoding and flag qubits, npj Quantum Inf. 6, 91 (2020).
  31. C. Chamberland, G. Zhu, T. J. Yoder, J. B. Hertzberg, and A. W. Cross, Topological and subsystem codes on low-degree graphs with flag qubits, Phys. Rev. X 10, 011022 (2020).
  32. H. Goto, Minimizing resource overheads for fault-tolerant preparation of encoded states of the Steane code, Sci. Rep. 6, 19578 (2016).
  33. A. Paetznick and B. W. Reichardt, Fault-tolerant ancilla preparation and noise threshold lower bounds for the 23-qubit Golay code, Quantum Inf. Comput. 12, 1034 (2012).
  34. F. Butt, S. Heußen, M. Rispler, and M. Müller, Fault-tolerant code-switching protocols for near-term quantum processors, PRX Quantum 5, 020345 (2024).
  35. H. Goto, Y. Ho, and T. Kanao, Measurement-free fault-tolerant logical-zero-state encoding of the distance-three nine-qubit surface code in a one-dimensional qubit array, Phys. Rev. Res. 5, 043137 (2023).
  36. C. Chamberland and A. W. Cross, Fault-tolerant magic state preparation with flag qubits, Quantum 3, 143 (2019).
  37. C. Ryan-Anderson, J. G. Bohnet, K. Lee, D. Gresh, A. Hankin, J. P. Gaebler, D. Francois, A. Chernoguzov, D. Lucchetti, N. C. Brown, T. M. Gatterman, S. K. Halit, K. Gilmore, J. A. Gerber, B. Neyenhuis, D. Hayes, and R. P. Stutz, Realization of real-time fault-tolerant quantum error correction, Phys. Rev. X 11, 041058 (2021).
  38. J. Hilder, D. Pijn, O. Onishchenko, A. Stahl, M. Orth, B. Lekitsch, A. Rodriguez-Blanco, M. Müller, F. Schmidt-Kaler, and U. G. Poschinger, Fault-tolerant parity readout on a shuttling-based trapped-ion quantum computer, Phys. Rev. X 12, 011032 (2022).
  39. I. Pogorelov, F. Butt, L. Postler, C. D. Marciniak, P. Schindler, M. Müller, and T. Monz, Experimental fault-tolerant code switching, arXiv:2403.13732.
  40. Qiskit transpiler, https://qiskit.org/documentation/apidoc/transpiler.html (accessed: 2024-01-17).
  41. F. Hua, M. Wang, G. Li, B. Peng, C. Liu, M. Zheng, S. Stein, Y. Ding, E. Z. Zhang, T. S. Humble, and A. Li, QASMTrans: A QASM based quantum transpiler framework for NISQ devices, arXiv:2308.07581.
  42. E. Younis and C. Iancu, Quantum circuit optimization and transpilation via parameterized circuit instantiation, in 2022 IEEE International Conference on Quantum Computing and Engineering (QCE) (IEEE, Broomfield, CO, USA, 2022), pp. 465–475.
  43. M. Maronese, L. Moro, L. Rocutto, and E. Prati, Quantum compiling, in Quantum Computing Environments, edited by S. S. Iyengar, M. Mastriani, and K. L. Kumar (Springer International Publishing, Cham, 2022), pp. 39–74.
  44. L. Schmid, D. F. Locher, M. Rispler, S. Blatt, J. Zeiher, M. Müller, and R. Wille, Computational capabilities and compiler development for neutral atom quantum processors—connecting tool developers and hardware experts, Quantum Sci. Technol. 9, 033001 (2024).
  45. S. Sivarajah, S. Dilkes, A. Cowtan, W. Simmons, A. Edgington, and R. Duncan, t|ket⟩: A retargetable compiler for NISQ devices, Quantum Sci. Technol. 6, 014003 (2021).
  46. N. Paraskevopoulos, F. Sebastiano, C. G. Almudever, and S. Feld, SpinQ: Compilation strategies for scalable spin-qubit architectures, arXiv:2301.13241.
  47. F. Kreppel, C. Melzer, D. O. Millán, J. Wagner, J. Hilder, U. Poschinger, F. Schmidt-Kaler, and A. Brinkmann, Quantum circuit compiler for a shuttling-based trapped-ion quantum computer, Quantum 7, 1176 (2023).
  48. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., Adaptive computation and machine learning series (The MIT Press, Cambridge, Massachusetts, 2018).
  49. M. Krenn, J. Landgraf, T. Foesel, and F. Marquardt, Artificial intelligence and machine learning for quantum technologies, Phys. Rev. A 107, 010101 (2023).
  50. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement learning with neural networks for quantum feedback, Phys. Rev. X 8, 031084 (2018).
  51. H. P. Nautrup, N. Delfosse, V. Dunjko, H. J. Briegel, and N. Friis, Optimizing quantum error correction codes with reinforcement learning, Quantum 3, 215 (2019).
  52. P. Andreasson, J. Johansson, S. Liljestrand, and M. Granath, Quantum error correction for the toric code using deep reinforcement learning, Quantum 3, 183 (2019).
  53. R. Sweke, M. S. Kesselring, E. P. L. Van Nieuwenburg, and J. Eisert, Reinforcement learning decoders for fault-tolerant quantum computation, Mach. Learn. 2, 025005 (2021).
  54. J. Olle, R. Zen, M. Puviani, and F. Marquardt, Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent, arXiv:2311.04750.
  55. M. Puviani, S. Borah, R. Zen, J. Olle, and F. Marquardt, Boosting the Gottesman-Kitaev-Preskill quantum error correction with non-Markovian feedback, arXiv:2312.07391.
  56. X.-M. Zhang, Z. Wei, R. Asad, X.-C. Yang, and X. Wang, When does reinforcement learning stand out in quantum control? A comparative study on state preparation, npj Quantum Inf. 5, 85 (2019).
  57. M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, Reinforcement learning in different phases of quantum control, Phys. Rev. X 8, 031086 (2018).
  58. R. Porotti, A. Essig, B. Huard, and F. Marquardt, Deep reinforcement learning for quantum state preparation with weak nonlinear measurements, Quantum 6, 747 (2022).
  59. T. Haug, W.-K. Mok, J.-B. You, W. Zhang, C. Eng Png, and L.-C. Kwek, Classifying global state preparation via deep reinforcement learning, Mach. Learn. 2, 01LT02 (2021).
  60. V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsioutsios, and M. H. Devoret, Model-free quantum control with reinforcement learning, Phys. Rev. X 12, 011059 (2022).
  61. S. Giordano and M. A. Martin-Delgado, Reinforcement-learning generation of four-qubit entangled states, Phys. Rev. Res. 4, 043056 (2022).
  62. L. Moro, M. G. A. Paris, M. Restelli, and E. Prati, Quantum compiling by deep reinforcement learning, Commun. Phys. 4, 178 (2021).
  63. Y.-H. Zhang, P.-L. Zheng, Y. Zhang, and D.-L. Deng, Topological quantum compiling with reinforcement learning, Phys. Rev. Lett. 125, 170501 (2020).
  64. Q. Chen, Y. Du, Q. Zhao, Y. Jiao, X. Lu, and X. Wu, Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning, arXiv:2204.06904.
  65. Z. He, L. Li, S. Zheng, Y. Li, and H. Situ, Variational quantum compiling with double Q-learning, New J. Phys. 23, 033002 (2021).
  66. W. Gong, S. Jiang, and D.-L. Deng, No-go theorem and a universal decomposition strategy for quantum channel compilation, Phys. Rev. Res. 5, 013060 (2023).
  67. L. M. Trenkwalder, E. Scerri, T. E. O’Brien, and V. Dunjko, Compilation of product-formula Hamiltonian simulation via reinforcement learning, arXiv:2311.04285.
  68. F. Preti, M. Schilling, S. Jerbi, L. M. Trenkwalder, H. P. Nautrup, F. Motzoi, and H. J. Briegel, Hybrid discrete-continuous compilation of trapped-ion quantum circuits with deep reinforcement learning, arXiv:2307.05744.
  69. S. Rietsch, A. Y. Dubey, C. Ufrecht, M. Periyasamy, A. Plinge, C. Mutschler, and D. D. Scherer, Unitary synthesis of Clifford+T circuits with reinforcement learning, in 2024 IEEE International Conference on Quantum Computing and Engineering (QCE) (2024), pp. 824–835, arXiv:2404.14865.
  70. D. Kremer, V. Villar, H. Paik, I. Duran, I. Faro, and J. Cruz-Benito, Practical and efficient quantum circuit synthesis and transpiling with reinforcement learning, arXiv:2405.13196.
  71. N. Shutty and C. Chamberland, Decoding merged color-surface codes and finding fault-tolerant Clifford circuits using solvers for satisfiability modulo theories, Phys. Rev. Appl. 18, 014072 (2022).
  72. D. Gottesman, Stabilizer codes and quantum error correction, arXiv:quant-ph/9705052.
  73. A. Steane, Multiple-particle interference and quantum error correction, Proc. R. Soc. A 452, 2551 (1996).
  74. A. R. Calderbank and P. W. Shor, Good quantum error-correcting codes exist, Phys. Rev. A 54, 1098 (1996).
  75. A. Y. Kitaev, Quantum computations: Algorithms and error correction, Russ. Math. Surv. 52, 1191 (1997).
  76. H. Bombín, Gauge color codes: Optimal transversal gates and gauge fixing in topological stabilizer codes, New J. Phys. 17, 083002 (2015).
  77. H. Bombin and M. A. Martin-Delgado, Topological quantum distillation, Phys. Rev. Lett. 97, 180501 (2006).
  78. S. Aaronson and D. Gottesman, Improved simulation of stabilizer circuits, Phys. Rev. A 70, 052328 (2004).
  79. S. Bravyi, J. A. Latone, and D. Maslov, 6-qubit optimal Clifford circuits, npj Quantum Inf. 8, 79 (2022).
  80. V. Kliuchnikov and D. Maslov, Optimization of Clifford circuits, Phys. Rev. A 88, 052307 (2013).
  81. S. Bravyi, R. Shaydulin, S. Hu, and D. Maslov, Clifford circuit optimization with templates and symbolic Pauli gates, Quantum 5, 580 (2021).
  82. S. Schneider, L. Burgholzer, and R. Wille, A SAT encoding for optimal Clifford circuit synthesis, in Proceedings of the 28th Asia and South Pacific Design Automation Conference (ACM, Tokyo Japan, 2023), pp. 190–195.
  83. D. Winder, Q. Huang, A. M.-v. de Griend, and R. Yeung, Architecture-aware synthesis of stabilizer circuits from Clifford tableaus, arXiv:2309.08972.
  84. P. Niemann, R. Wille, and R. Drechsler, Efficient synthesis of quantum circuits implementing Clifford group operations, in 2014 19th Asia and South Pacific Design Automation Conference (ASP-DAC) (IEEE, Singapore, 2014), pp. 483–488.
  85. D. Amaro, M. Müller, and A. K. Pal, Scalable characterization of localizable entanglement in noisy topological quantum codes, New J. Phys. 22, 053038 (2020).
  86. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, 10th ed. (Cambridge University Press, Cambridge; New York, 2010).
  87. N. Rengaswamy, R. Calderbank, S. Kadhe, and H. D. Pfister, Logical Clifford synthesis for stabilizer codes, IEEE Trans. Quantum Eng. 1, 1 (2020).
  88. A. Mondal and K. K. Parhi, Optimization of quantum circuits for stabilizer codes, IEEE Trans. Circuits Syst. I 71, 3647 (2024).
  89. D. Tandeitnik and T. Guerreiro, Evolving quantum circuits, Quantum Inf. Process. 23, 109 (2024).
  90. A. M. Steane, Overhead and noise threshold of fault-tolerant quantum error correction, Phys. Rev. A 68, 042322 (2003).
  91. C. Chamberland, A. Kubica, T. J. Yoder, and G. Zhu, Triangular color codes on trivalent graphs with flag qubits, New J. Phys. 22, 023019 (2020).
  92. E. Knill, Quantum computing with realistically noisy devices, Nature (London) 434, 39 (2005).
  93. T. Tansuwannont and D. Leung, Achieving fault tolerance on capped color codes with few ancillas, PRX Quantum 3, 030322 (2022).
  94. K. C. Miao et al., Overcoming leakage in quantum error correction, Nat. Phys. 19, 1780 (2023).
  95. C. Ryan-Anderson et al., High-fidelity and fault-tolerant teleportation of a logical qubit using transversal gates and lattice surgery on a trapped-ion quantum computer, arXiv:2404.16728.
  96. H. Bombín, M. Pant, S. Roberts, and K. I. Seetharam, Fault-tolerant post-selection for low overhead magic state preparation (2022).
  97. L. Egan, D. M. Debroy, C. Noel, A. Risinger, D. Zhu, D. Biswas, M. Newman, M. Li, K. R. Brown, M. Cetina, and C. Monroe, Fault-tolerant operation of a quantum error-correction code, arXiv:2009.11482.
  98. R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, arXiv:1706.06643.
  99. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
  100. T. Fösel, M. Y. Niu, F. Marquardt, and L. Li, Quantum circuit optimization with deep reinforcement learning, arXiv:2103.07585.
  101. M. Ostaszewski, L. M. Trenkwalder, W. Masarczyk, E. Scerri, and V. Dunjko, Reinforcement learning for optimization of variational quantum circuit architectures, Adv. Neural Inf. Process. Syst. 34, 18182 (2021).
  102. T. Gabor, M. Zorn, and C. Linnhoff-Popien, The applicability of reinforcement learning for the automatic generation of state preparation circuits, in Proceedings of the Genetic and Evolutionary Computation Conference Companion (ACM, Boston Massachusetts, 2022), pp. 2196–2204.
  103. X. Xu, S. C. Benjamin, and X. Yuan, Variational circuit compiler for quantum error correction, Phys. Rev. Appl. 15, 034068 (2021).
  104. J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, Barren plateaus in quantum neural network training landscapes, Nat. Commun. 9, 4812 (2018).
  105. C. Lu, J. G. Kuba, A. Letcher, L. Metz, C. S. de Witt, and J. Foerster, Discovered policy optimisation, Adv. Neural Inf. Process. Syst. 35, 16455 (2022).
  106. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, JAX: Composable transformations of python+numpy programs (2018).
  107. https://github.com/remmyzen/rlftqc.
  108. S. Z. Baba, N. Yoshioka, Y. Ashida, and T. Sagawa, Deep reinforcement learning for preparation of thermal and prethermal quantum states, Phys. Rev. Appl. 19, 014068 (2023).
  109. J. Mackeprang, D. B. R. Dasari, and J. Wrachtrup, A reinforcement learning approach for quantum state engineering, Quantum Mach. Intell. 2, 5 (2020).
  110. R. Laflamme, C. Miquel, J. P. Paz, and W. H. Zurek, Perfect quantum error correcting code, Phys. Rev. Lett. 77, 198 (1996).
  111. P. W. Shor, Scheme for reducing decoherence in quantum computer memory, Phys. Rev. A 52, R2493 (1995).
  112. N. H. Nguyen, M. Li, A. M. Green, C. Huerta Alderete, Y. Zhu, D. Zhu, K. R. Brown, and N. M. Linke, Demonstration of Shor encoding on a trapped-ion quantum computer, Phys. Rev. Appl. 16, 024057 (2021).
  113. R. Zhang, L.-Z. Liu, Z.-D. Li, Y.-Y. Fei, X.-F. Yin, L. Li, N.-L. Liu, Y. Mao, Y.-A. Chen, and J.-W. Pan, Loss-tolerant all-photonic quantum repeater with generalized Shor code, Optica 9, 152 (2022).
  114. D. Nigg, M. Müller, E. A. Martinez, P. Schindler, M. Hennrich, T. Monz, M. A. Martin-Delgado, and R. Blatt, Quantum computations on a topologically encoded qubit, Science 345, 302 (2014).
  115. D. Bluvstein, H. Levine, G. Semeghini, T. T. Wang, S. Ebadi, M. Kalinowski, A. Keesling, N. Maskara, H. Pichler, M. Greiner, V. Vuletić, and M. D. Lukin, A quantum processor based on coherent transport of entangled atom arrays, Nature (London) 604, 451 (2022).
  116. J. T. Anderson, G. Duclos-Cianci, and D. Poulin, Fault-tolerant conversion between the Steane and Reed-Muller quantum codes, Phys. Rev. Lett. 113, 080501 (2014).
  117. J. I. Cirac and P. Zoller, Quantum computations with cold trapped ions, Phys. Rev. Lett. 74, 4091 (1995).
  118. Qiskit contributors, Qiskit: An open-source framework for quantum computing (2023), 10.5281/zenodo.2573505.
  119. FakeManilaV2, https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.fake_provider.FakeManilaV2 (accessed: 2024−02−20).
  120. FakeJakartaV2, https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.fake_provider.FakeJakartaV2 (accessed: 2024-02-20).
  121. FakeGuadalupeV2, https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.fake_provider.FakeGuadalupeV2 (accessed: 2024-02-20).
  122. FakeTokyo, https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.fake_provider.FakeTokyo (accessed: 2024-02-20).
  123. M. E. Taylor and P. Stone, Transfer learning for reinforcement learning domains: A survey, J. Mach. Learn. Res. 10 (2009).
  124. Z. Zhu, K. Lin, A. K. Jain, and J. Zhou, Transfer learning in deep reinforcement learning: A survey, IEEE Trans. Pattern Anal. Mach. Intell. 45, 13344 (2023).
  125. C. Gidney, Stim: A fast stabilizer circuit simulator, Quantum 5, 497 (2021).
  126. F. Arute, K. Arya, R. Babbush et al., Quantum supremacy using a programmable superconducting processor, Nature (London) 574, 505 (2019).
  127. R. S. Gupta, N. Sundaresan, T. Alexander, C. J. Wood, S. T. Merkel, M. B. Healy, M. Hillenbrand, T. Jochym-O’Connor, J. R. Wootton, T. J. Yoder, A. W. Cross, M. Takita, and B. J. Brown, Encoding a magic state with beyond break-even fidelity, Nature (London) 625, 259 (2024).
  128. A. G. Fowler, M. Mariantoni, J. M. Martinis, and A. N. Cleland, Surface codes: Towards practical large-scale quantum computation, Phys. Rev. A 86, 032324 (2012).
  129. Y. Ye et al., Logical magic state preparation with fidelity beyond the distillation threshold on a superconducting quantum processor, Phys. Rev. Lett. 131, 210603 (2023).
  130. S. Heußen, L. Postler, M. Rispler, I. Pogorelov, C. D. Marciniak, T. Monz, P. Schindler, and M. Müller, Strategies for a practical advantage of fault-tolerant circuit design in noisy trapped-ion quantum computers, Phys. Rev. A 107, 042422 (2023).
  131. Y. Tomita and K. M. Svore, Low-distance surface codes under realistic quantum noise, Phys. Rev. A 90, 062320 (2014).
  132. R. Acharya et al., Quantum error correction below the surface code threshold, arXiv:2408.13687.
  133. N. Berthusen, J. Dreiling, C. Foltz, J. P. Gaebler, T. M. Gatterman, D. Gresh, N. Hewitt, M. Mills, S. A. Moses, B. Neyenhuis, P. Siegfried, and D. Hayes, Experiments with the 4D surface code on a QCCD quantum computer, Phys. Rev. A 110, 062413 (2024).
  134. B. W. Reichardt, D. Aasen, R. Chao, A. Chernoguzov, W. v. Dam, J. P. Gaebler, D. Gresh, D. Lucchetti, M. Mills, S. A. Moses, B. Neyenhuis, A. Paetznick, A. Paz, P. E. Siegfried, M. P. d. Silva, K. M. Svore, Z. Wang, and M. Zanner, Demonstration of quantum computation and error correction with a tesseract code, arXiv:2409.04628.
  135. B. Hetényi and J. R. Wootton, Creating entangled logical qubits in the heavy-hex lattice with topological codes, PRX Quantum 5, 040334 (2024).
  136. B. W. Reichardt et al., Logical computation demonstrated with a neutral atom quantum processor, arXiv:2411.11822.
  137. N. Delfosse and B. W. Reichardt, Short Shor-style syndrome sequences, arXiv:2008.05051.
  138. D. Bhatnagar, M. Steinberg, D. Elkouss, C. G. Almudever, and S. Feld, Low-depth flag-style syndrome extraction for small quantum error-correction codes, in 2023 IEEE International Conference on Quantum Computing and Engineering (QCE) (IEEE, Bellevue, WA, 2023), pp. 63–69.
  139. S. Kosub, A note on the triangle inequality for the Jaccard distance, arXiv:1612.02696.
  140. FakeLimaV2, https://docs.quantum.ibm.com/api/qiskit-ibm-runtime/qiskit_ibm_runtime.fake_provider.FakeLimaV2 (accessed: 2024-02-20).
  141. E. Knill and R. Laflamme, Concatenated quantum codes, Technical Report LA-UR–96-2808, 369608, 1996.
  142. M. Grassl, P. Shor, G. Smith, J. Smolin, and B. Zeng, Generalized concatenated quantum codes, Phys. Rev. A 79, 050306(R) (2009).
  143. D. Gottesman, Surviving as a quantum computer in a classical world (2024), preprint on webpage at https://www.cs.umd.edu/class/spring2024/cmsc858G.
  144. S. Gronauer and K. Diepold, Multi-agent deep reinforcement learning: A survey, Artif. Intell. Rev. 55, 895 (2022).
  145. S. V. Albrecht, F. Christianos, and L. Schäfer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches (MIT Press, Cambridge, MA, 2024).

Outline

Information

Phys. Rev. X 15, 041012– Published 22 October, 2025
  • Received 17 May 2024
  • Revised 7 March 2025
  • Accepted 11 September 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Open access publication funded by the Max Planck Society.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation