- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

From tensor-network quantum states to tensorial recurrent neural networks

Dian Wu*, Riccardo Rossi†, Filippo Vicentini‡, and Giuseppe Carleo§

  • Institute of Physics, École Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland

  • *dian.wu@epfl.ch
  • †riccardo.rossi@epfl.ch
  • ‡filippo.vicentini@epfl.ch
  • §giuseppe.carleo@epfl.ch
PDF

Phys. Rev. Research 5, L032001 – Published 5 July, 2023

DOI: https://doi.org/10.1103/PhysRevResearch.5.L032001

Abstract

We show that any matrix product state (MPS) can be exactly represented by a recurrent neural network (RNN) with a linear memory update. We generalize this RNN architecture to two-dimensional lattices using a multilinear memory update. It supports perfect sampling and wave-function evaluation in polynomial time, and can represent an area law of entanglement entropy. Numerical evidence shows that it can encode the wave function using a bond dimension lower by orders of magnitude when compared to MPS, with an accuracy that can be systematically improved by increasing the bond dimension.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (57)

  1. R. J. Baxter, Dimers on a rectangular lattice, J. Math. Phys. 9, 650 (1968).
  2. I. Affleck, T. Kennedy, E. H. Lieb, and H. Tasaki, Rigorous Results on Valence-Bond Ground States in Antiferromagnets, Phys. Rev. Lett. 59, 799 (1987).
  3. H. Takasaki, T. Hikihara, and T. Nishino, Fixed point of the finite system DMRG, J. Phys. Soc. Jpn. 68, 1537 (1999).
  4. F. Verstraete and J. I. Cirac, Matrix product states represent ground states faithfully, Phys. Rev. B 73, 094423 (2006).
  5. M. B. Hastings, An area law for one-dimensional quantum systems, J. Stat. Mech.: Theory Exp. (2007) P08024.
  6. S. R. White, Density Matrix Formulation for Quantum Renormalization Groups, Phys. Rev. Lett. 69, 2863 (1992).
  7. N. Schuch, M. M. Wolf, F. Verstraete, and J. I. Cirac, Computational Complexity of Projected Entangled Pair States, Phys. Rev. Lett. 98, 140506 (2007).
  8. G. Lami, G. Carleo, and M. Collura, Matrix product states with backflow correlations, Phys. Rev. B 106, L081111 (2022).
  9. G. Evenbly, Algorithms for tensor network renormalization, Phys. Rev. B 95, 045117 (2017).
  10. F. Pan, P. Zhou, S. Li, and P. Zhang, Contracting Arbitrary Tensor Networks: General Approximate Algorithm and Applications in Graphical Models and Quantum Circuit Simulations, Phys. Rev. Lett. 125, 060503 (2020).
  11. L. Vanderstraeten, L. Burgelman, B. Ponsioen, M. Van Damme, B. Vanhecke, P. Corboz, J. Haegeman, and F. Verstraete, Variational methods for contracting projected entangled-pair states, Phys. Rev. B 105, 195140 (2022).
  12. A. W. Sandvik and G. Vidal, Variational Quantum Monte Carlo Simulations with Tensor-Network States, Phys. Rev. Lett. 99, 220602 (2007).
  13. A. Sfondrini, J. Cerrillo, N. Schuch, and J. I. Cirac, Simulating two- and three-dimensional frustrated quantum systems with string-bond states, Phys. Rev. B 81, 214426 (2010).
  14. W.-Y. Liu, Y.-Z. Huang, S.-S. Gong, and Z.-C. Gu, Accurate simulation for finite projected entangled pair states in two dimensions, Phys. Rev. B 103, 235155 (2021).
  15. T. Vieijra, J. Haegeman, F. Verstraete, and L. Vanderstraeten, Direct sampling of projected entangled-pair states, Phys. Rev. B 104, 235141 (2021).
  16. E. Stoudenmire and D. J. Schwab, Supervised learning with tensor networks, in Advances in Neural Information Processing Systems, Vol. 29 (Curran Associates, Red Hook, NY, 2016), pp. 4799–4807.
  17. S. Cheng, L. Wang, and P. Zhang, Supervised learning with projected entangled pair states, Phys. Rev. B 103, 125117 (2021).
  18. Z.-Y. Han, J. Wang, H. Fan, L. Wang, and P. Zhang, Unsupervised Generative Modeling Using Matrix Product States, Phys. Rev. X 8, 031012 (2018).
  19. J. Liu, S. Li, J. Zhang, and P. Zhang, Tensor networks for unsupervised machine learning, Phys. Rev. E 107, L012103 (2023).
  20. S. Cheng, L. Wang, T. Xiang, and P. Zhang, Tree tensor networks for generative modeling, Phys. Rev. B 99, 155131 (2019).
  21. T. Vieijra, L. Vanderstraeten, and F. Verstraete, Generative modeling with projected entangled-pair states, arXiv:2202.08177.
  22. I. Sutskever, J. Martens, and G. E. Hinton, Generating text with recurrent neural networks, Proceedings of the 28th International Conference on Machine Learning (Omnipress, Madison, WI, 2011), pp. 1017–1024.
  23. O. Irsoy and C. Cardie, Modeling compositionality with multiplicative recurrent neural networks, 3rd International Conference on Learning Representations (ACM, New York, 2015).
  24. M. Hibat-Allah, E. M. Inack, R. Wiersema, R. G. Melko, and J. Carrasquilla, Variational neural annealing, Nat. Mach. Intell. 3, 952 (2021).
  25. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  26. G. Carleo, Y. Nomura, and M. Imada, Constructing exact representations of quantum many-body systems with deep neural networks, Nat. Commun. 9, 5322 (2018).
  27. D.-L. Deng, X. Li, and S. Das Sarma, Machine learning topological states, Phys. Rev. B 96, 195145 (2017).
  28. I. Glasser, N. Pancotti, M. August, I. D. Rodriguez, and J. I. Cirac, Neural-Network Quantum States, String-Bond States, and Chiral Topological States, Phys. Rev. X 8, 011006 (2018).
  29. R. Kaubruegger, L. Pastori, and J. C. Budich, Chiral topological phases from artificial neural networks, Phys. Rev. B 97, 195136 (2018).
  30. S. Lu, X. Gao, and L.-M. Duan, Efficient representation of topologically ordered states with restricted Boltzmann machines, Phys. Rev. B 99, 155136 (2019).
  31. J. Chen, S. Cheng, H. Xie, L. Wang, and T. Xiang, Equivalence of restricted Boltzmann machines and tensor network states, Phys. Rev. B 97, 085104 (2018).
  32. L. Pastori, R. Kaubruegger, and J. C. Budich, Generalized transfer matrix states from artificial neural networks, Phys. Rev. B 99, 165123 (2019).
  33. H. Poon and P. Domingos, Sum-product networks: A new deep architecture, IEEE International Conference on Computer Vision Workshops (IEEE Computer Society, Washington, DC, 2011), pp. 689–690.
  34. Y. Levine, O. Sharir, N. Cohen, and A. Shashua, Quantum Entanglement in Deep Learning Architectures, Phys. Rev. Lett. 122, 065301 (2019).
  35. V. Khrulkov, A. Novikov, and I. Oseledets, Expressive power of recurrent neural networks, 6th International Conference on Learning Representations (ACM, New York, 2018).
  36. O. Sharir, A. Shashua, and G. Carleo, Neural tensor contractions and the expressive power of deep neural quantum states, Phys. Rev. B 106, 205136 (2022).
  37. M. Hibat-Allah, M. Ganahl, L. E. Hayward, R. G. Melko, and J. Carrasquilla, Recurrent neural network wave functions, Phys. Rev. Res. 2, 023358 (2020).
  38. A. J. Ferris and G. Vidal, Perfect sampling with unitary tensor networks, Phys. Rev. B 85, 165146 (2012).
  39. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.5.L032001 for mathematical definitions of our tensorial recurrent neural network architectures, proofs for the computational complexity and the entanglement entropy, details of numerical experiments, and additional figures visualizing the properties of variational states, which includes Refs. [52, 53, 54, 55, 56, 57].
  40. E. M. Stoudenmire and S. R. White, Studying two-dimensional systems with the density matrix renormalization group, Annu. Rev. Condens. Matter Phys. 3, 111 (2012).
  41. Y. Levine, O. Sharir, A. Ziv, and A. Shashua, On the long-term memory of deep recurrent networks, arXiv:1710.09431.
  42. A. W. Sandvik, Finite-size scaling of the ground-state parameters of the two-dimensional Heisenberg model, Phys. Rev. B 56, 11678 (1997).
  43. W.-Y. Liu, S.-J. Dong, Y.-J. Han, G.-C. Guo, and L. He, Gradient optimization of finite projected entangled pair states, Phys. Rev. B 95, 195154 (2017).
  44. W. Marshall, Antiferromagnetism, Proc. R. Soc. London, Ser. A 232, 48 (1955).
  45. A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, Automatic differentiation in machine learning: A survey, J. Mach. Learn. Res. 18, 1 (2018).
  46. O. Sharir, Y. Levine, N. Wies, G. Carleo, and A. Shashua, Deep Autoregressive Models for the Efficient Variational Simulation of Many-Body Quantum Systems, Phys. Rev. Lett. 124, 020503 (2020).
  47. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, 3rd International Conference on Learning Representations (Ref. [23]).
  48. T. Westerhout, N. Astrakhantsev, K. S. Tikhonov, M. I. Katsnelson, and A. A. Bagrov, Generalization properties of neural network approximations to frustrated magnet ground states, Nat. Commun. 11, 1593 (2020).
  49. M. Bukov, M. Schmitt, and M. Dupont, Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape, SciPost Phys. 10, 147 (2021).
  50. S.-S. Gong, W. Zhu, D. N. Sheng, O. I. Motrunich, and M. P. A. Fisher, Plaquette Ordered Phase and Quantum Phase Diagram in the Spin-1/2J1−J2 Square Heisenberg Model, Phys. Rev. Lett. 113, 027201 (2014).
  51. https://github.com/cqsl/mps-rnn.
  52. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9, 1735 (1997).
  53. K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio, On the properties of neural machine translation: Encoder-decoder approaches, Proceedings of SSST@EMNLP 2014, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (Association for Computational Linguistics, Stroudsburg, PA, 2014), pp. 103–111.
  54. L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika 31, 279 (1966).
  55. G. Carleo, K. Choo, D. Hofmann, J. E. T. Smith, T. Westerhout, F. Alet, E. J. Davis, S. Efthymiou, I. Glasser, S.-H. Lin, M. Mauri, G. Mazzola, C. B. Mendl, E. van Nieuwenburg, O. O'Reilly, H. Théveniaut, G. Torlai, F. Vicentini, and A. Wietek, NetKet: A machine learning toolkit for many-body quantum systems, SoftwareX 10, 100311 (2019).
  56. F. Vicentini, D. Hofmann, A. Szabó, D. Wu, C. Roth, C. Giuliani, G. Pescia, J. Nys, V. Vargas-Calderon, N. Astrakhantsev, and G. Carleo, NetKet 3: Machine learning toolbox for many-body quantum systems, SciPost Phys. Codebases, 7 (2022), doi:10.21468/SciPostPhysCodeb.7.
  57. M. Fishman, S. R. White, and E. M. Stoudenmire, The ITensor software library for tensor network calculations, SciPost Phys. Codebases, 4 (2022), doi:10.21468/SciPostPhysCodeb.4.

Outline

Information

Phys. Rev. Research 5, L032001– Published 5 July, 2023
  • Received 20 March 2023
  • Accepted 9 May 2023
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation