- Physical Review Journals
All JournalsPhysics Magazine

PRX Intelligence

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Transformer Neural-Network Quantum States for Lattice Models of Spins and Fermions: Application to the Ancilla Layer Model

Riccardo Rende1,*,†, Alexander Nikolaenko2,*,‡, Luciano Loris Viteritti3, Subir Sachdev1,4, and Ya-Hui Zhang5

  • *These authors contributed equally to this work.
  • †Contact author: rrende@flatironinstitute.org
  • ‡Contact author: onikolaienko@g.harvard.edu
PDF

PRX Intelligence 1, 023007 – Published 22 October, 2026

DOI: https://doi.org/10.1103/swyd-jq37

Abstract

We introduce a variational wave function based on neural-network quantum states (NQS) to study lattice systems whose local Hilbert space contains both spin and fermionic degrees of freedom. Our approach is based on the use of the transformer architecture, which can naturally handle composite local Hilbert spaces through a tokenization procedure closely inspired by techniques from natural language processing. The key methodological innovation is that the neural network predicts a set of fermionic orbitals that depend on the spin configuration in a backflow-inspired manner. We apply the method to the one-dimensional ancilla layer model, consisting of a chain of mobile spin-1/2 fermions coupled to a two-leg spin-1/2 ladder. For open boundary conditions, we achieve excellent quantitative agreement with density matrix renormalization group results across the full range of parameters considered. We find a phase in which the chain forms an effectively decoupled Luttinger liquid (LL), and an LL* phase with a distinct Fermi wave vector in which the mobile fermions are Kondo screened by one leg of the ladder, while the other leg forms the critical Bethe spin liquid. The LL* is the analog of the FL* phase in two dimensions proposed to describe the cuprate pseudogap. We also find a Luther-Emery phase, where the LL* state becomes unstable toward the formation of a spin gap. The transformer ansatz maintains comparable accuracy for periodic boundary conditions, where tensor-network methods are computationally more demanding. Moreover, we show that the approach extends naturally to fully periodic two-dimensional clusters, a regime beyond the reach of current tensor-network methods. Together, these findings establish transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight its potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (91)

  1. S. R. White, Density matrix formulation for quantum renormalization groups, Phys. Rev. Lett. 69, 2863 (1992).
  2. W. M. C. Foulkes, L. Mitas, R. J. Needs, and G. Rajagopal, Quantum Monte Carlo simulations of solids, Rev. Mod. Phys. 73, 33 (2001).
  3. A. W. Sandvik, Computational studies of quantum spin systems, AIP Conf. Proc. 1297, 135 (2010).
  4. F. Verstraete, V. Murg, and J. Cirac, Matrix product states, projected entangled pair states, and variational renormalization group methods for quantum spin systems, Adv. Phys. 57, 143 (2008).
  5. U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Ann. Phys. 326, 96 (2011).
  6. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  7. H. Lange, A. Van de Walle, A. Abedinnia, and A. Bohrdt, From architectures to applications: A review of neural quantum states, Quantum Sci. Technol. 9, 040501 (2024).
  8. R. Rende, L. L. Viteritti, L. Bardone, F. Becca, and S. Goldt, A simple linear algebra identity to optimize large-scale neural network quantum states, Commun. Phys. 7, 260 (2024).
  9. L. L. Viteritti, R. Rende, S. Sachdev, and G. Carleo, Approaching the thermodynamic limit with neural-network quantum states, arXiv:2602.02665.
  10. Y. Gu, W. Li, H. Lin, B. Zhan, R. Li, Y. Huang, D. He, Y. Wu, T. Xiang, M. Qin, L. Wang, and D. Lv, Solving the Hubbard model with neural quantum states, Nat. Commun. 17, 7838 (2026).
  11. C. Roth, A. Chen, A. Sengupta, and A. Georges, Superconductivity in the two-dimensional Hubbard model revealed by neural quantum states, arXiv:2511.07566.
  12. A. Chen and M. Heyl, Empowering deep neural quantum states through efficient optimization, Nat. Phys. 20, 1476 (2024).
  13. J. Hubbard, Electron correlations in narrow energy bands, Proc. R. Soc. London A 276, 238 (1963).
  14. J. Kanamori, Electron correlation and ferromagnetism of transition metals, Prog. Theor. Phys. 30, 275 (1963).
  15. S. Doniach, The Kondo lattice and weak antiferromagnetism, Physica B+C 91, 231 (1977).
  16. A. Georges, L. de’ Medici, and J. Mravlje, Strong correlations from Hund's coupling, Annu. Rev. Condens. Matter Phys. 4, 137 (2013).
  17. T. Li, P. Wölfle, and P. J. Hirschfeld, Spin-rotation-invariant slave-boson approach to the Hubbard model, Phys. Rev. B 40, 6817 (1989).
  18. L. de’ Medici, S. R. Hassan, M. Capone, and X. Dai, Orbital-selective Mott transition out of band degeneracy lifting, Phys. Rev. Lett. 102, 126401 (2009).
  19. Y. Nomura, Machine learning quantum states—Extensions to fermion–boson coupled systems and excited-state calculations, J. Phys. Soc. Jpn. 89, 054706 (2020).
  20. V. Marino, A. Scazzola, F. Becca, M. Capone, and L. F. Tocchio, Intertwined superconductivity and orbital selectivity in a three-orbital Hubbard model for the iron pnictides, Phys. Rev. Lett. 134, 196502 (2025).
  21. D. Piccioni, F. Ferrari, M. Fabrizio, and F. Becca, Insulating and metallic phases in the one-dimensional Hubbard-Su-Schrieffer-Heeger model: Insights from a backflow-inspired variational wave function, Phys. Rev. B 111, 045125 (2025).
  22. R. Favata, D. Piccioni, A. Parola, and F. Becca, Interaction-induced phases in the half-filled Bernevig-Hughes-Zhang model in one dimension, Phys. Rev. B 111, 155105 (2025).
  23. A. Mahajan, P. J. Robinson, J. Lee, and D. R. Reichman, Structure and dynamics of electron-phonon coupled systems using neural quantum states, Phys. Rev. B 112, 184310 (2025).
  24. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems, edited by I. Guyon, U. Von Luxburg, S. Bengio, H.Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Vol. 30 (Curran Associates, Inc., 2017).
  25. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al., Transformers: State-of-the-art natural language processing, in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, edited by Q. Liu and D. Schlangen (Association for Computational Linguistics, 2020), pp. 38–45.
  26. T. Lin, Y. Wang, X. Liu, and X. Qiu, A survey of transformers, AI Open 3, 111 (2022).
  27. L. L. Viteritti, R. Rende, and F. Becca, Transformer variational wave functions for frustrated quantum spin systems, Phys. Rev. Lett. 130, 236401 (2023).
  28. K. Sprague and S. Czischek, Variational Monte Carlo with large patched transformers, Commun. Phys. 7, 90 (2024).
  29. L. L. Viteritti, R. Rende, A. Parola, S. Goldt, and F. Becca, Transformer wave function for two dimensional frustrated magnets: Emergence of a spin-liquid phase in the Shastry-Sutherland model, Phys. Rev. B 111, 134411 (2025).
  30. L. L. Viteritti, R. Rende, G. B. Testasecca, J. Niedda, R. Moessner, G. Carleo, and A. Scardicchio, Quantum spin glass in the two-dimensional disordered Heisenberg model via foundation neural-network quantum states, arXiv:2507.05073.
  31. R. Rende, L. L. Viteritti, F. Becca, A. Scardicchio, A. Laio, and G. Carleo, Foundation neural-networks quantum states as a unified ansatz for multiple Hamiltonians, Nat. Commun. 16, 7213 (2025).
  32. I. von Glehn, J. S. Spencer, and D. Pfau, A self-attention ansatz for ab initio quantum chemistry, arXiv:2211.13672.
  33. L. Sharma, A. Shokry, R. Nutakki, O. Simard, M. Ferrero, and F. Vicentini, Comparing symmetrized determinant neural quantum states for the Hubbard model, Phys. Rev. B 113, 245104, (2026).
  34. H. Ma, B. Kan, H. Shang, and J. Yang, Transformer-based neural networks backflow for strongly correlated electronic structure, arXiv:2509.25720.
  35. H. Shang, C. Guo, Y. Wu, Z. Li, others, and J. Yang, Solving the many-electron Schrödinger equation with a transformer-based framework, Nat. Commun. 16, 8464 (2025).
  36. Y. Teng, D. D. Dai, and L. Fu, Solving the fractional quantum Hall problem with self-attention neural network, Phys. Rev. B 111, 205117 (2025).
  37. D. Jurafsky and J. H. Martin, Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition, with language models (2026), 3rd ed., https://web.stanford.edu/~jurafsky/slp3/.
  38. A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions, Rev. Mod. Phys. 68, 13 (1996).
  39. P. M. Bonetti, M. Christos, A. Nikolaenko, A. A. Patel, and S. Sachdev, Fractionalized Fermi liquids and the cuprate phase diagram, Rep. Prog. Phys. 89, 044501 (2026).
  40. A. Nikolaenko, M. Tikhanovskaya, S. Sachdev, and Y.-H. Zhang, Small to large Fermi surface transition in a single-band model using randomly coupled ancillas, Phys. Rev. B 103, 235138 (2021).
  41. E. Mascot, A. Nikolaenko, M. Tikhanovskaya, Y.-H. Zhang, D. K. Morr, and S. Sachdev, Electronic spectra with paramagnon fractionalization in the single-band Hubbard model, Phys. Rev. B 105, 075146 (2022).
  42. V. J. Emery, S. A. Kivelson, and O. Zachar, Spin-gap proximity effect mechanism of high-temperature superconductivity, Phys. Rev. B 56, 6120 (1997).
  43. J. Voit, One-dimensional Fermi liquids, Rep. Prog. Phys. 58, 977 (1995).
  44. Y.-H. Zhang and S. Sachdev, From the pseudogap metal to the Fermi liquid using ancilla qubits, Phys. Rev. Res. 2, 023172 (2020).
  45. Y.-H. Zhang and S. Sachdev, Deconfined criticality and ghost Fermi surfaces at the onset of antiferromagnetism in a metal, Phys. Rev. B 102, 155124 (2020).
  46. A. Nikolaenko, J. von Milczewski, D. G. Joshi, and S. Sachdev, Spin density wave, Fermi liquid, and fractionalized phases in a theory of antiferromagnetic metals using paramagnons and bosonic spinons, Phys. Rev. B 108, 045123 (2023).
  47. M. Christos, Z.-X. Luo, H. Shackleton, Y.-H. Zhang, M. S. Scheurer, and S. Sachdev, A model of d-wave superconductivity, antiferromagnetism, and charge order on the square lattice, Proc. Natl. Acad. Sci. USA 120, e2302701120 (2023).
  48. M. Christos and S. Sachdev, Emergence of nodal Bogoliubov quasiparticles across the transition from the pseudogap metal to the d-wave superconductor, npj Quantum Mater. 9, 4 (2024).
  49. P. M. Bonetti, M. Christos, and S. Sachdev, Quantum oscillations in the hole-doped cuprates and the confinement of spinons, Proc. Natl. Acad. Sci. USA 121, e2418633121 (2024).
  50. H. Pandey, M. Christos, P. M. Bonetti, R. Shanker, A. Nikolaenko, S. Sharma, and S. Sachdev, Thermal SU(2) lattice gauge theory of the cuprate pseudogap: Reconciling Fermi arcs and hole pockets, Proc. Natl. Acad. Sci. USA 123, e2606117123 (2026).
  51. P. Coleman, A. Panigrahi, and A. Tsvelik, A microscopic model of fractionalized Fermi liquid, arXiv:2511.01115.
  52. P. Coleman, E. J. König, A. Panigrahi, and A. Tsvelik, Tractable model for a fractionalized Fermi liquid (FL*) on a square lattice, Phys. Rev. B 114, 125126 (2026).
  53. N. Read and D. M. Newns, On the solution of the Coqblin-Schreiffer Hamiltonian by the large-n expansion technique, J. Phys. C 16, 3273 (1983).
  54. P. Coleman, Mixed valence as an almost broken symmetry, Phys. Rev. B 35, 5072 (1987).
  55. A. Nikolaenko and Y.-H. Zhang, Numerical signatures of ultra-local criticality in a one dimensional Kondo lattice model, SciPost Phys. 17, 034 (2024).
  56. F. Verstraete, D. Porras, and J. I. Cirac, Density matrix renormalization group and periodic boundary conditions: A quantum information perspective, Phys. Rev. Lett. 93, 227205 (2004).
  57. A. W. Sandvik and G. Vidal, Variational quantum Monte Carlo simulations with tensor-network states, Phys. Rev. Lett. 99, 220602 (2007).
  58. P. Pippan, S. R. White, and H. G. Evertz, Efficient matrix-product state method for periodic boundary conditions, Phys. Rev. B 81, 081103 (2010).
  59. K. Choo, G. Carleo, N. Regnault, and T. Neupert, Symmetries and many-body excitations with neural-network quantum states, Phys. Rev. Lett. 121, 167204 (2018).
  60. Y. Nomura and M. Imada, Dirac-type nodal spin liquid revealed by refined quantum many-body solver using neural-network wave function, correlation ratio, and level spectroscopy, Phys. Rev. X 11, 031034 (2021).
  61. L. Viteritti, F. Ferrari, and F. Becca, Accuracy of restricted Boltzmann machines for the one-dimensional J1−J2 Heisenberg model, SciPost Phys. 12, 166 (2022).
  62. E. Stoudenmire and S. R. White, Studying two-dimensional systems with the density matrix renormalization group, Annu. Rev. Condens. Matter Phys. 3, 111 (2012).
  63. R. Orús, A practical introduction to tensor networks: Matrix product states and projected entangled pair states, Ann. Phys. 349, 117 (2014).
  64. A. W. Sandvik, Finite-size scaling of the ground-state parameters of the two-dimensional Heisenberg model, Phys. Rev. B 56, 11678 (1997).
  65. S. Jelassi, M. Sander, and Y. Li, Vision transformers provably learn spatial structure, in Advances in Neural Information Processing Systems, edited by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Curran Associates, Inc., Red Hook, NY, 2022), Vol. 35, pp. 37822–37836.
  66. R. Rende, F. Gerace, A. Laio, and S. Goldt, Mapping of attention mechanisms to a generalized Potts model, Phys. Rev. Res. 6, 023057 (2024).
  67. R. Rende and L. Loris Viteritti, Are queries and keys always relevant? A case study on transformer wave functions, Mach. Learn.: Sci. Technol. 6, 010501 (2025).
  68. R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T.-Y. Liu, On layer normalization in the transformer architecture, arXiv:2002.04745.
  69. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, arXiv:1512.03385.
  70. D. Luo and B. K. Clark, Backflow transformations via neural networks for quantum many-body wave functions, Phys. Rev. Lett. 122, 226401 (2019).
  71. R. Rende, A. Sinibaldi, L. L. Viteritti, R. Wiersema, A. Georges, and G. Carleo, Scaling laws for neural-network quantum states, arXiv:2606.02794.
  72. F. Becca, W.-J. Hu, Y. Iqbal, A. Parola, D. Poilblanc, and S. Sorella, Lanczos steps to improve variational wave functions, J. Phys.: Conf. Ser. 640, 012039 (2015).
  73. T. Barnes, E. Dagotto, J. Riera, and E. S. Swanson, Excitation spectrum of Heisenberg spin ladders, Phys. Rev. B 47, 3196 (1993).
  74. I. P. McCulloch, Infinite size density matrix renormalization group, revisited, arXiv:0804.2509.
  75. F. Pollmann, S. Mukerjee, A. M. Turner, and J. E. Moore, Theory of finite-entanglement scaling at one-dimensional quantum critical points, Phys. Rev. Lett. 102, 255701 (2009).
  76. P. Calabrese and J. Cardy, Entanglement entropy and quantum field theory, J. Stat. Mech. (2004) P07013.
  77. Y.-H. Zhang and Z. Zhu, Fractional Fermi liquid in a generalized t–j model, Phys. Rev. B 103, 115101 (2021).
  78. E. Lieb, T. Schultz, and D. Mattis, Two soluble models of an antiferromagnetic chain, Ann. Phys. 16, 407 (1961).
  79. M. Yamanaka, M. Oshikawa, and I. Affleck, Nonperturbative approach to Luttinger's theorem in one dimension, Phys. Rev. Lett. 79, 1110 (1997).
  80. T. Senthil, S. Sachdev, and M. Vojta, Fractionalized Fermi liquids, Phys. Rev. Lett. 90, 216403 (2003).
  81. E. Berg, E. Fradkin, and S. A. Kivelson, Pair-density-wave correlations in the Kondo-Heisenberg model, Phys. Rev. Lett. 105, 146403 (2010).
  82. Y.-H. Zhang and A. Vishwanath, Pair-density-wave superconductor from doping Haldane chain and rung-singlet ladder, Phys. Rev. B 106, 045103 (2022).
  83. Y. Nomura, Helping restricted Boltzmann machines with quantum-state representation by restoring symmetry, J. Phys.: Condens. Matter 33, 174003 (2021).
  84. L. Zhang and D. Luo, Neural transformer backflow for solving momentum-resolved ground states of strongly correlated materials, arXiv:2509.09275.
  85. P. Coleman, Heavy Fermions: Electrons at the Edge of Magnetism, in Handbook of Magnetism and Advanced Magnetic Materials (John Wiley and Sons, Ltd, New York, NY, 2007).
  86. B. Danu, Z. Liu, F. F. Assaad, and M. Raczkowski, Zooming in on heavy fermions in Kondo lattice models, Phys. Rev. B 104, 155128 (2021).
  87. A. W. Sandvik and D. J. Scalapino, Order-disorder transition in a two-layer quantum antiferromagnet, Phys. Rev. Lett. 72, 2777 (1994).
  88. F. Becca and S. Sorella, Quantum Monte Carlo Approaches for Correlated Systems (Cambridge University Press, Cambridge, England, 2017).
  89. S. Sorella, Green function Monte Carlo with stochastic reconfiguration, Phys. Rev. Lett. 80, 4558 (1998).
  90. G. Goldshlager, N. Abrahamsen, and L. Lin, A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions, J. Comput. Phys. 516, 113351 (2024).
  91. J. Hauschild and F. Pollmann, Efficient numerical simulations with tensor networks: Tensor Network Python (TeNPy), SciPost Phys. Lect. Notes 5 (2018).

Outline

Information

PRX Intelligence 1, 023007– Published 22 October, 2026
  • Received 5 March 2026
  • Accepted 8 September 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation