- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

Yuto Sugimoto*

  • *Contact author: sugimoto@nucl.phys.tohoku.ac.jp
PDF

Phys. Rev. D 113, 094502 – Published 8 May, 2026

DOI: https://doi.org/10.1103/9wb1-rrcn

Abstract

We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group methods. In this approach, evaluating the derivatives of the partition function up to the order of k increases the matrix-multiplication cost by a factor of (k+1)(k+2)/2 compared to computing the free energy alone, and the memory footprint is only k+1 times that of the original calculation. In the limit where the derivatives of the singular value decomposition are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in tensor renormalization group calculations in both two and three dimensions. In addition, we discuss how to efficiently differentiate an arbitrary tensor network.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (89)

  1. M. Levin and C. P. Nave, Phys. Rev. Lett. 99, 120601 (2007).
  2. Z.-C. Gu and X.-G. Wen, Phys. Rev. B 80, 155131 (2009).
  3. Z. Y. Xie, J. Chen, M. P. Qin, J. W. Zhu, L. P. Yang, and T. Xiang, Phys. Rev. B 86, 045139 (2012).
  4. Y. Shimizu and Y. Kuramashi, Phys. Rev. D 90, 074503 (2014).
  5. J. F. Unmuth-Yockey, Y. Meurice, J. Osborn, and H. Zou, Proc. Sci. LATTICE2014 (2014) 325 [arXiv:1411.4213].
  6. J. F. Yu, Z. Y. Xie, Y. Meurice, Y. Liu, A. Denbleyker, H. Zou, M. P. Qin, and J. Chen, Phys. Rev. E 89, 013308 (2014).
  7. S. Wang, Z. Y. Xie, J. Chen, B. Normand, and T. Xiang, Chin. Phys. Lett. 31, 070503 (2014).
  8. Y. Shimizu and Y. Kuramashi, Phys. Rev. D 90, 014508 (2014).
  9. S. Takeda and Y. Yoshimura, Prog. Theor. Exp. Phys. 2015, 043B01 (2015).
  10. S. Takeda and Y. Yoshimura, Prog. Theor. Exp. Phys. 2015, 043B01 (2015).
  11. H. Kawauchi and S. Takeda, Phys. Rev. D 93, 114503 (2016).
  12. R. Sakai, S. Takeda, and Y. Yoshimura, Prog. Theor. Exp. Phys. 2017, 063B07 (2017).
  13. R. Sakai, D. Kadoh, Y. Kuramashi, Y. Nakamura, S. Takeda, and Y. Yoshimura, EPJ Web Conf. 175, 11019 (2018).
  14. Y. Yoshimura, Y. Kuramashi, Y. Nakamura, S. Takeda, and R. Sakai, Phys. Rev. D 97, 054511 (2018).
  15. A. Bazavov, S. Catterall, R. G. Jha, and J. Unmuth-Yockey, Phys. Rev. D 99, 114507 (2019).
  16. S. Iino, S. Morita, and N. Kawashima, Phys. Rev. B 100, 035449 (2019).
  17. S. Hong and D.-H. Kim, Phys. Rev. E 101, 012124 (2020).
  18. S. Akiyama, Y. Kuramashi, T. Yamashita, and Y. Yoshimura, Phys. Rev. D 100, 054510 (2019).
  19. S. Akiyama, D. Kadoh, Y. Kuramashi, T. Yamashita, and Y. Yoshimura, J. High Energy Phys. 09 (2020) 177.
  20. C. Delcamp and A. Tilloy, Phys. Rev. Res. 2, 033278 (2020).
  21. S. Akiyama and Y. Kuramashi, Phys. Rev. D 104, 014504 (2021).
  22. S. Akiyama, Y. Kuramashi, T. Yamashita, and Y. Yoshimura, J. High Energy Phys. 01 (2021) 121.
  23. S. Akiyama, Y. Kuramashi, and Y. Yoshimura, Phys. Rev. D 104, 034507 (2021).
  24. J. Bloch, R. G. Jha, R. Lohmayer, and M. Meister, Phys. Rev. D 104, 094517 (2021).
  25. M. Fukuma, D. Kadoh, and N. Matsumoto, Prog. Theor. Exp. Phys. 2021, 123B03 (2021).
  26. S. Akiyama and D. Kadoh, J. High Energy Phys. 10 (2021) 188.
  27. M. Hirasawa, A. Matsumoto, J. Nishimura, and A. Yosprakob, J. High Energy Phys. 12 (2021) 011.
  28. S. Akiyama, Y. Kuramashi, and T. Yamashita, Prog. Theor. Exp. Phys. 2022, 023I01 (2022).
  29. G. Li, K. H. Pai, and Z.-C. Gu, Phys. Rev. Res. 4, 023159 (2022).
  30. K. Nakayama, L. Funcke, K. Jansen, Y.-J. Kao, and S. Kühn, Phys. Rev. D 105, 054507 (2022).
  31. S. Akiyama and Y. Kuramashi, J. High Energy Phys. 05 (2022) 102.
  32. J. Bloch, R. Lohmayer, and M. Meister, Proc. Sci. LATTICE2021 (2022) 285.
  33. T. Kuwahara and A. Tsuchiya, Prog. Theor. Exp. Phys. 2022, 093B02 (2022).
  34. X. Luo and Y. Kuramashi, Phys. Rev. D 107, 094509 (2023).
  35. J. Bloch and R. Lohmayer, Nucl. Phys. B986, 116032 (2023).
  36. M. Asaduzzaman, S. Catterall, Y. Meurice, R. Sakai, and G. C. Toga, J. High Energy Phys. 01 (2023) 024.
  37. K. Yoshiyama and K. Hukushima, Phys. Rev. E 108, 054124 (2023).
  38. J. Genzor, Phys. Rev. E 107, 034131 (2023).
  39. S. Akiyama and Y. Kuramashi, J. High Energy Phys. 10 (2023) 77.
  40. S. Akiyama, R. G. Jha, and J. Unmuth-Yockey, Phys. Rev. D 110, 034519 (2024).
  41. S. Akiyama and Y. Kuramashi, J. High Energy Phys. 09 (2024) 086.
  42. K. Nakayama and M. Schneider, Phys. Rev. D 110, 094501 (2024).
  43. A. Samlodia, V. Longia, R. G. Jha, and A. Joseph, Phys. Rev. D 110, 034504 (2024).
  44. K. Homma, S. Morita, and N. Kawashima, Phys. Rev. B 111, 134427 (2025).
  45. Y. Sugimoto, S. Akiyama, and Y. Kuramashi, Phys. Rev. D 113, 034503 (2026).
  46. H. Aizawa, S. Takeda, and Y. Yoshimura, Prog. Theor. Exp. Phys. 2026, 013B07 (2026).
  47. H. Kanno, S. Akiyama, K. Murakami, and S. Takeda, J. High Energy Phys. 11 (2025) 036.
  48. S. Morita, R. Igarashi, H.-H. Zhao, and N. Kawashima, Phys. Rev. E 97, 033310 (2018).
  49. D. Adachi, T. Okubo, and S. Todo, Phys. Rev. B 102, 054432 (2020).
  50. D. Kadoh and K. Nakayama, arXiv:1912.02414.
  51. D. Adachi, T. Okubo, and S. Todo, Phys. Rev. B 105, L060402 (2022).
  52. G. Evenbly and G. Vidal, Phys. Rev. Lett. 115, 180405 (2015).
  53. S. Yang, Z.-C. Gu, and X.-G. Wen, Phys. Rev. Lett. 118, 110504 (2017).
  54. M. Hauru, C. Delcamp, and S. Mizera, Phys. Rev. B 97, 045111 (2018).
  55. S. Morita and N. Kawashima, Phys. Rev. B 103, 045131 (2021).
  56. K. Homma, T. Okubo, and N. Kawashima, Phys. Rev. Res. 6, 043102 (2024).
  57. F.-F. Song and N. Kawashima, Phys. Rev. B 112, 235167 (2025).
  58. S. Morita and N. Kawashima, Comput. Phys. Commun. 236, 65 (2019).
  59. H.-J. Liao, J.-G. Liu, L. Wang, and T. Xiang, Phys. Rev. X 9, 031041 (2019).
  60. S. Morita and N. Kawashima, Phys. Rev. B 111, 054433 (2025).
  61. M. Bartholomew-Biggs, S. Brown, B. Christianson, and L. Dixon, J. Comput. Appl. Math. 124, 171 (2000).
  62. A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, J. Mach. Learn. Res. 18, 1 (2018), https://www.jmlr.org/papers/v18/17-468.html.
  63. B.-B. Chen, Y. Gao, Y.-B. Guo, Y. Liu, H.-H. Zhao, H.-J. Liao, L. Wang, T. Xiang, W. Li, and Z. Y. Xie, Phys. Rev. B 101, 220409 (2020).
  64. C. Geng, H.-Y. Hu, and Y. Zou, Mach. Learn. Sci. Technol. 3, 015020 (2022).
  65. Y.-H. Tseng, S.-W. Li, and F.-J. Jiang, Prog. Theor. Exp. Phys. 2025, 073A01 (2025).
  66. https://github.com/wangleiphy/tensorgrad (2021).
  67. https://github.com/under-Peter/TensorNetworkAD.jl (2023).
  68. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, USA, 2019), Vol. 32, https://proceedings.neurips.cc/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html.
  69. M. Innes, arXiv:1810.07951.
  70. F. I. Az-zahra, S. Takeda, and T. Yamazaki, Phys. Rev. D 110, 034514 (2024).
  71. L. Wang and F. Verstraete, arXiv:1110.4362.
  72. S. Iino, S. Morita, and N. Kawashima, Phys. Rev. B 101, 155418 (2020).
  73. G. Evenbly and R. N. C. Pfeifer, Phys. Rev. B 89, 245118 (2014).
  74. K. Nakayama and M. Schneider, Phys. Rev. D 110, 094501 (2024).
  75. G. Evenbly, Phys. Rev. B 95, 045117 (2017).
  76. Y. Nakamura, H. Oba, and S. Takeda, Phys. Rev. B 99, 155101 (2019).
  77. https://datarepo.mdcl.issp.u-tokyo.ac.jp/repo/49 (2026).
  78. S. Morita and N. Kawashima, J. Phys. Soc. Jpn. 95, 044001 (2026).
  79. S. Akiyama, R. G. Jha, J. Maeda, Y. Tanizaki, and J. Unmuth-Yockey, Phys. Rev. D 113, 074502 (2026).
  80. A. Ueda and M. Oshikawa, Phys. Rev. B 108, 024413 (2023).
  81. F. Kos, D. Poland, D. Simmons-Duffin, and A. Vichi, J. High Energy Phys. 08 (2016) 036.
  82. S. R. White, Phys. Rev. B 48, 10345 (1993).
  83. A. Francuz, N. Schuch, and B. Vanhecke, Phys. Rev. Res. 7, 013237 (2025).
  84. L. Devos and J. Haegeman, arXiv:2508.10076.
  85. A. Ueda, S. D. Meyer, A. Naravane, V. Vanthilt, and F. Verstraete, arXiv:2508.05406.
  86. K. Nakayama, arXiv:2307.14191.
  87. J. Townsend, Differentiating the singular value decomposition, technical report, 2016, https://j-towns.github.io/papers/svd-derivative.pdf, 2016.
  88. A. A. Gorodetsky, C. Safta, and J. D. Jakeman, J. Mach. Learn. Res. 23, 1 (2022), https://www.jmlr.org/papers/v23/21-0225.html.
  89. J. Bettencourt, M. J. Johnson, and D. Duvenaud, in Program Transformations for ML Workshop at NeurIPS 2019 (2019), https://openreview.net/forum?id=SkxEF3FNPH.

Outline

Information

Phys. Rev. D 113, 094502– Published 8 May, 2026
  • Received 11 February 2026
  • Accepted 10 April 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation