- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Design principles of deep translationally symmetric neural quantum states for frustrated magnets

Rajah P. Nutakki*, Ahmedeo Shokry, and Filippo Vicentini

  • CPHT, CNRS, École Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France; Collège de France, Université PSL, 11 place Marcelin Berthelot, 75005 Paris, France; Inria Paris-Saclay, 91120 Palaiseau, France; and LIX, CNRS, Ecole Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France

  • *Contact author: rajah.nutakki@polytechnique.edu
PDF

Phys. Rev. Research 7, 043099 – Published 24 October, 2025

DOI: https://doi.org/10.1103/ybgv-35jm

Abstract

Deep neural-network quantum states have emerged as a leading method for studying the ground states of quantum magnets. Successful architectures exploit translational symmetry, but the root of their effectiveness and differences between architectures remain unclear. Here, we apply the ConvNext architecture, designed to incorporate elements of transformers into convolutional networks, to quantum many-body ground states. We find that it is remarkably similar to the factored vision transformer, which has been employed successfully for several frustrated spin systems, allowing us to relate this architecture to more conventional convolutional networks. Through a series of numerical experiments, we design the ConvNext to achieve greatest performance at lowest computational cost, and then apply this network to the Shastry-Sutherland and J1−J2 models, obtaining variational energies comparable to the state of the art, providing a blueprint for network design choices of translationally symmetric architectures to tackle challenging ground-state problems in frustrated magnetism.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (80)

  1. S. R. White, Density matrix formulation for quantum renormalization groups, Phys. Rev. Lett. 69, 2863 (1992).
  2. S. R. White, Density-matrix algorithms for quantum renormalization groups, Phys. Rev. B 48, 10345 (1993).
  3. U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Ann. Phys. 326, 96 (2011).
  4. E. M. Stoudenmire and S. R. White, Studying two-dimensional systems with the density matrix renormalization group, Annu. Rev. Condens. Matter Phys. 3, 111 (2012).
  5. S.-J. Ran, E. Tirrito, C. Peng, X. Chen, L. Tagliacozzo, G. Su, and M. Lewenstein, Tensor Network Contractions: Methods and Applications to Quantum Many-Body Systems, Lecture Notes in Physics (Springer International Publishing, Cham, 2020), Vol. 964.
  6. N. V. Prokof'ev, B. V. Svistunov, and I. S. Tupitsyn, Exact, complete, and universal continuous-time worldline Monte Carlo approach to the statistics of discrete quantum systems, J. Exp. Theor. Phys. 87, 310 (1998).
  7. A. W. Sandvik, Stochastic series expansion method with operator-loop update, Phys. Rev. B 59, R14157 (1999).
  8. O. F. Syljuåsen and A. W. Sandvik, Quantum Monte Carlo with directed loops, Phys. Rev. E 66, 046701 (2002).
  9. H. G. Evertz, The loop algorithm, Adv. Phys. 52, 1 (2003).
  10. K. V. Houcke, E. Kozik, N. Prokof'ev, and B. Svistunov, Diagrammatic Monte Carlo, Phys. Proc. 6, 95 (2010).
  11. F. Becca and S. Sorella, Quantum Monte Carlo Approaches for Correlated Systems, 1st ed (Cambridge University Press, Cambridge, UK, 2017).
  12. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  13. L. L. Viteritti, R. Rende, A. Parola, S. Goldt, and F. Becca, Transformer wave function for two dimensional frustrated magnets: Emergence of a spin-liquid phase in the Shastry-Sutherland model, Phys. Rev. B 111, 134411 (2025).
  14. T. Duric, J. H. Chung, B. Yang, and P. Sengupta, Spin-1/2 kagome Heisenberg antiferromagnet: Machine learning discovery of the spinon pair density wave ground state, Phys. Rev. X 15, 011047 (2025).
  15. A. Chen, Empowering deep neural quantum states through efficient optimization, Nat. Phys. 20, 1476 (2024).
  16. R. Rende, L. L. Viteritti, L. Bardone, F. Becca, and S. Goldt, A simple linear algebra identity to optimize large-scale neural network quantum states, Commun. Phys. 7, 260 (2024).
  17. C. Roth, A. Szabó, and A. H. MacDonald, High-accuracy variational Monte Carlo for frustrated magnets with deep neural networks, Phys. Rev. B 108, 054410 (2023).
  18. M. Li, J. Chen, Q. Xiao, F. Wang, Q. Jiang, X. Zhao, R. Lin, H. An, X. Liang, and L. He, Bridging the gap between deep learning and frustrated quantum spin system for extreme-scale simulations on new generation of sunway supercomputer, IEEE Trans. Parallel Distrib. Syst. 33, 2846 (2022).
  19. X. Liang, M. Li, Q. Xiao, J. Chen, C. Yang, H. An, and L. He, Deep learning representations for quantum many-body systems on heterogeneous hardware, Mach. Learn.: Sci. Technol. 4, 015035 (2023).
  20. L. Gravina, V. Savona, and F. Vicentini, Neural projected quantum dynamics: A systematic study, Quantum 9, 1803 (2025).
  21. L. Mauron, Z. Denis, J. Nys, and G. Carleo, Predicting topological entanglement entropy in a Rydberg analog simulator, Nat. Phys. 21, 1332 (2025).
  22. T. Mendes-Santos, M. Schmitt, and M. Heyl, Highly resolved spectral functions of two-dimensional systems with neural quantum states, Phys. Rev. Lett. 131, 046501 (2023).
  23. S. Sorella, Green function Monte Carlo with stochastic reconfiguration, Phys. Rev. Lett. 80, 4558 (1998).
  24. S.-I. Amari, Natural gradient works efficiently in learning, Neural Comput. 10, 251 (1998).
  25. J. Martens, New insights and perspectives on the natural gradient method, J. Mach. Learn. Res. 21, 1 (2020).
  26. S. Dash, L. Gravina, F. Vicentini, M. Ferrero, and A. Georges, Efficiency of neural quantum states in light of the quantum geometric tensor, Commun. Phys. 8, 92 (2025).
  27. R. Rende, F. Gerace, A. Laio, and S. Goldt, Mapping of attention mechanisms to a generalized Potts model, Phys. Rev. Res. 6, 023057 (2024).
  28. L. L. Viteritti, R. Rende, and F. Becca, Transformer variational wave functions for frustrated quantum spin systems, Phys. Rev. Lett. 130, 236401 (2023).
  29. R. Rende and L. L. Viteritti, Are queries and keys always relevant? A case study on transformer wave functions, Mach. Learn.: Sci. Technol. 6, 010501 (2024).
  30. A. Chen, V. D. Naik, and M. Heyl, Convolutional transformer wave functions, arXiv:2503.10462.
  31. Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, A ConvNet for the 2020s, in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New Orleans, LA, 2022) pp. 11966–11976.
  32. S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, ConvNeXt V2: Co-designing and scaling ConvNets with masked autoencoders, in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, Vancouver, BC, Canada, 2023), pp. 16133–16142.
  33. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems (Curran Associates, Inc., 2017), Vol. 30.
  34. S. Islam, H. Elmekki, A. Elsebai, J. Bentahar, N. Drawel, G. Rjoub, and W. Pedrycz, A comprehensive survey on applications of transformers for deep learning tasks, Expert Syst. Appl. 241, 122666 (2024).
  35. B. Sriram Shastry and B. Sutherland, Exact ground state of a quantum mechanical antiferromagnet, Physica B+C 108, 1069 (1981).
  36. Introduction to frustrated magnetism, in Springer Series in Solid-State Sciences, edited by C. Lacroix, P. Mendels, and F. Mila (Springer, Heidelberg, 2011), Vol. 164.
  37. J. Yang, A. W. Sandvik, and L. Wang, Quantum criticality and spin liquid phase in the Shastry-Sutherland model, Phys. Rev. B 105, L060409 (2022).
  38. L. Wang, Y. Zhang, and A. W. Sandvik, Quantum spin liquid phase in the Shastry-Sutherland model detected by an improved level spectroscopic method, Chin. Phys. Lett. 39, 077502 (2022).
  39. A. Keleş and E. Zhao, Rise and fall of plaquette order in the Shastry-Sutherland magnet revealed by pseudofermion functional renormalization group, Phys. Rev. B 105, L041115 (2022).
  40. L. Wang and A. W. Sandvik, Critical level crossings and gapless spin liquid in the square-lattice spin-1/2 J1 − J2 Heisenberg antiferromagnet, Phys. Rev. Lett. 121, 107202 (2018).
  41. F. Ferrari and F. Becca, Gapless spin liquid and valence-bond solid in the J1 − J2 Heisenberg model on the square lattice: Insights from singlet and triplet excitations, Phys. Rev. B 102, 014417 (2020).
  42. Y. Nomura and M. Imada, Dirac-type nodal spin liquid revealed by refined quantum many-body solver using neural-network wave function, correlation ratio, and level spectroscopy, Phys. Rev. X 11, 031034 (2021).
  43. W.-Y. Liu, S.-S. Gong, Y.-B. Li, D. Poilblanc, W.-Q. Chen, and Z.-C. Gu, Gapless quantum spin liquid and global phase diagram of the spin-1/2 J1 − J2 square antiferromagnetic Heisenberg model, Sci. Bull. 67, 1034 (2022).
  44. W.-J. Hu, F. Becca, A. Parola, and S. Sorella, Direct evidence for a gapless Z2 spin liquid by frustrating Néel antiferromagnetism, Phys. Rev. B 88, 060402 (2013).
  45. F. Ferrari and F. Becca, Spectral signatures of fractionalization in the frustrated Heisenberg model on the square lattice, Phys. Rev. B 98, 100405 (2018).
  46. A. Maity, F. Ferrari, J. Y. Lee, J. Potten, T. Müller, R. Thomale, R. Samajdar, and Y. Iqbal, Evidence for a Z2 Dirac spin liquid in the generalized Shastry-Sutherland model, arXiv:2501.00096.
  47. K. Choo, T. Neupert, and G. Carleo, Two-dimensional frustrated J1 − J2 model studied with neural network quantum states, Phys. Rev. B 100, 125124 (2019).
  48. A. Szabó and C. Castelnovo, Neural network wave functions and the sign problem, Phys. Rev. Res. 2, 033075 (2020).
  49. Z. Denis, A. Sinibaldi, and G. Carleo, Comment on “can neural quantum states learn volume-law ground states?” Phys. Rev. Lett. 134, 079701 (2025).
  50. A. Sinibaldi, C. Giuliani, G. Carleo, and F. Vicentini, Unbiasing time-dependent variational Monte Carlo by projected quantum evolution, Quantum 7, 1131 (2023).
  51. W. Marshall, Antiferromagnetism, Proc. R. Soc. Lond. Ser. A 232, 48 (1955).
  52. The translation of the input gives a corresponding translation of the output.
  53. The encoder is not equivariant, and the whole network not translationally symmetric, under subpatch translations. For example, considering the square lattice with 2×2 patches of Fig. 5, the network would not transform correctly under translations by ONE site along the x or y axes.
  54. A. G. Wilson, Position: Deep learning is not so Mysterious or different, in Forty-second International Conference on Machine Learning Position Paper Track (ICML, 2025).
  55. K. Sprague and S. Czischek, Variational Monte Carlo with large patched transformers, Commun. Phys. 7, 90 (2024).
  56. M. Schmitt and M. Heyl, Quantum many-body dynamics in two dimensions with artificial neural networks, Phys. Rev. Lett. 125, 100503 (2020).
  57. R. Rende, S. Goldt, F. Becca, and L. L. Viteritti, Fine-tuning neural network quantum states, Phys. Rev. Res. 6, 043280 (2024).
  58. T. Mizusaki and M. Imada, Quantum-number projection in the path-integral renormalization group method, Phys. Rev. B 69, 125110 (2004).
  59. Y. Nomura, Helping restricted Boltzmann machines with quantum-state representation by restoring symmetry, J. Phys.: Condens. Matter 33, 174003 (2021).
  60. M. Hibat-Allah, M. Ganahl, L. E. Hayward, R. G. Melko, and J. Carrasquilla, Recurrent neural network wave functions, Phys. Rev. Res. 2, 023358 (2020).
  61. M. Hibat-Allah, R. G. Melko, and J. Carrasquilla, Supplementing recurrent neural network wave functions with symmetry and annealing to improve accuracy, in 35th Annual Conference on Neural Information Processing Systems (NeurIPS, Vancouver, BC, Canada, 2021).
  62. M. Hibat-Allah, R. G. Melko, and J. Carrasquilla, Investigating topological order using recurrent neural networks, Phys. Rev. B 108, 075152 (2023).
  63. M. S. Moss, R. Wiersema, M. Hibat-Allah, J. Carrasquilla, and R. G. Melko, Leveraging recurrence in neural network wavefunctions for large-scale simulations of Heisenberg antiferromagnets: The square lattice, arXiv:2502.17144.
  64. D. S. Kufel, J. Kemp, D. Vu, S. M. Linsel, C. R. Laumann, and N. Y. Yao, Approximately-symmetric neural networks for quantum spin liquids, Phys. Rev. Lett. 135, 056702 (2025).
  65. Y. Nomura, A. S. Darmawan, Y. Yamaji, and M. Imada, Restricted Boltzmann machine learning for solving strongly correlated quantum systems, Phys. Rev. B 96, 205152 (2017).
  66. S. Morita, R. Kaneko, and M. Imada, Quantum spin liquid in spin 1/2J1−J2 Heisenberg model on square lattice: Many-variable variational Monte Carlo study combined with quantum-number projections, J. Phys. Soc. Jpn. 84, 024720 (2015).
  67. A. Sinibaldi, D. Hendry, F. Vicentini, and G. Carleo, Time-dependent neural Galerkin method for quantum dynamics, arXiv:2412.11778.
  68. A. V. de Walle, M. Schmitt, and A. Bohrdt, Many-body dynamics with explicitly time-dependent neural quantum states, Mach. Learn.: Sci. Technol. 6, 045011 (2025).
  69. F. Vicentini, D. Hofmann, A. Szabó, D. Wu, C. Roth, C. Giuliani, G. Pescia, J. Nys, V. Vargas-Calderón, N. Astrakhantsev, and G. Carleo, NetKet 3: Machine learning toolbox for many-body quantum systems, SciPost Phys. Codebases 7 (2022).
  70. G. Carleo, K. Choo, D. Hofmann, J. E. T. Smith, T. Westerhout, F. Alet, E. J. Davis, S. Efthymiou, I. Glasser, S.-H. Lin, M. Mauri, G. Mazzola, C. B. Mendl, E. van Nieuwenburg, O. O’Reilly, H. Théveniaut, G. Torlai, F. Vicentini, and A. Wietek, NetKet: A machine learning toolkit for many-body quantum systems, SoftwareX 10, 100311 (2019).
  71. D. Häfner and F. Vicentini, mpi4jax: Zero-copy MPI communication of JAX arrays, J. Open Source Softw. 65, 3419 (2021).
  72. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, JAX: Composable transformations of Python+Numpy programs, GitHub (2018), https://github.com/jax-ml/jax.
  73. J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Rondepierre, A. Steiner, and M. van Zee, Flax: A neural network library and ecosystem for JAX, GitHub (2024), http://github.com/google/flax.
  74. Convnext neural quantum states and lattice symmetries, GitHub (2025), https://github.com/NeuralQXLab/convnextnnqs.
  75. R. P. Nutakki, A. Shokry, and F. Vicentini, Data for “Design principles of deep translationally symmetric neural quantum states for frustrated magnets”, Zenodo (2025), https://doi.org/10.5281/zenodo.17157844.
  76. G. Goldshlager, N. Abrahamsen, and L. Lin, A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions, J. Comput. Phys. 516, 113351 (2024).
  77. M. Reh, M. Schmitt, and M. Gärttner, Optimizing design choices for neural quantum states, Phys. Rev. B 107, 195115 (2023).
  78. Z. Denis and G. Carleo, Accurate neural quantum states for interacting lattice bosons, Quantum 9, 1772 (2025).
  79. Vision transformer wave function (2025), https://netket.readthedocs.io/en/stable/tutorials/ViT-wave-function.html.
  80. I. Loshchilov and F. Hutter, SGDR: Stochastic gradient descent with warm restarts, in International Conference on Learning Representations (ICLR, 2017).

Outline

Information

Phys. Rev. Research 7, 043099– Published 24 October, 2025
  • Received 13 May 2025
  • Accepted 15 September 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation