- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Synaptic field theory for neural networks

Donghee Lee*, Hye-Sung Lee†, and Jaeok Yi‡

  • *Contact author: dhlee641@kaist.ac.kr
  • †Contact author: hyesung.lee@kaist.ac.kr
  • ‡Contact author: wodhr1541@kaist.ac.kr
PDF

Phys. Rev. D 112, L031902 – Published 15 August, 2025

DOI: https://doi.org/10.1103/bhwt-j2rq

Abstract

Theoretical understanding of deep learning remains elusive despite its empirical success. In this study, we propose a novel “synaptic field theory” that describes the training dynamics of synaptic weights and biases in the continuum limit. Unlike previous approaches, our framework treats synaptic weights and biases as fields and interprets their indices as spatial coordinates, with the training data acting as external sources. This perspective offers new insights into the fundamental mechanisms of deep learning and suggests a pathway for leveraging well-established field-theoretic techniques to study neural network training.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (74)

  1. Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature (London) 521, 436 (2015).
  2. F.-L. Fan, J. Xiong, M. Li, and G. Wang, On interpretability of artificial neural networks: A survey, IEEE Trans. Radiat. Plasma Med. Sci. 5, 741 (2021).
  3. X. Li, H. Xiong, X. Li, X. Wu, X. Zhang, J. Liu, J. Bian, and D. Dou, Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond, Knowl. Inf. Syst. 64, 3197 (2022).
  4. Y. Bahri, J. Kadmon, J. Pennington, S. S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annu. Rev. Condens. Matter Phys. 11, 501 (2020).
  5. S. Krippendorf and M. Spannowsky, A duality connecting neural network and cosmological dynamics, Mach. Learn. Sci. Tech. 3, 035011 (2022).
  6. K. Hashimoto, S. Sugishita, A. Tanaka, and A. Tomiya, Deep learning and the AdS/CFT correspondence, Phys. Rev. D 98, 046019 (2018).
  7. K. Hashimoto, S. Sugishita, A. Tanaka, and A. Tomiya, Deep learning and holographic QCD, Phys. Rev. D 98, 106014 (2018).
  8. K. Hashimoto, AdS/CFT correspondence as a deep Boltzmann machine, Phys. Rev. D 99, 106017 (2019).
  9. W.-C. Gan and F.-W. Shu, Holography as deep learning, Int. J. Mod. Phys. D 26, 1743020 (2017).
  10. J. Lee, Y. Bahri, R. Novak, S. S. Schoenholz, J. Pennington, and J. Sohl-Dickstein, Deep neural networks as Gaussian processes, arXiv:1711.00165.
  11. J. M. Antognini, Finite size corrections for neural network Gaussian processes, arXiv:1908.10030.
  12. B. Hanin and M. Nica, Finite depth and width corrections to the neural tangent kernel, arXiv:1909.05989.
  13. J. Huang and H.-T. Yau, Dynamics of deep neural networks and neural tangent hierarchy, arXiv:1909.08156.
  14. E. Dyer and G. Gur-Ari, Asymptotics of wide networks from Feynman diagrams, arXiv:1909.11304.
  15. Y. Sho, Non-Gaussian processes and neural networks at finite widths, arXiv:1910.00019.
  16. G. Naveh, O. Ben David, H. Sompolinsky, and Z. Ringel, Predicting the outputs of finite deep neural networks trained with noisy gradients, Phys. Rev. E 104, 064301 (2021).
  17. I. Seroussi, G. Naveh, and Z. Ringel, Separation of scales and a thermodynamic description of feature learning in some CNNS, arXiv:2112.15383.
  18. K. Aitken and G. Gur-Ari, On the asymptotics of wide networks with polynomial activations, arXiv:2006.06687.
  19. A. Andreassen and E. Dyer, Asymptotics of wide convolutional neural networks, arXiv:2008.08675.
  20. J. Halverson, A. Maiti, and K. Stoner, Neural networks and quantum field theory, Mach. Learn. Sci. Tech. 2, 035002 (2021).
  21. J. A. Zavatone-Veth, A. Canatar, B. S. Ruben, and C. Pehlevan, Asymptotics of representation learning in finite Bayesian neural networks, J. Stat. Mech. (2022) 114008.
  22. G. Naveh and Z. Ringel, A self-consistent theory of Gaussian processes captures feature learning effects in finite CNNs, in Adv. Neural Inf. Process. Syst. (2021), Vol. 34, p. 21352.
  23. B. Hanin, Random fully connected neural networks as perturbatively solvable hierarchies, arXiv:2204.01058.
  24. S. Yaida, Meta-principled family of hyperparameter scaling strategies, arXiv:2210.04909.
  25. M. Demirtas, J. Halverson, A. Maiti, M. D. Schwartz, and K. Stoner, Neural network field theories: Non-gaussianity, actions, and locality, Mach. Learn. Sci. Technol. 5, 015002 (2024).
  26. H. Erbin, V. Lahoche, and D. O. Samary, Non-perturbative renormalization for the neural network-QFT correspondence, Mach. Learn. Sci. Tech. 3, 015027 (2022).
  27. H. Erbin, V. Lahoche, and D. O. Samary, Renormalization in the neural network-quantum field theory correspondence, arXiv:2212.11811.
  28. Z. Ringel, N. Rubin, E. Mor, M. Helias, and I. Seroussi, Applications of statistical field theory in deep learning, arXiv:2502.18553.
  29. M. A. Buice and C. C. Chow, Beyond mean field theory: Statistical field theory for neural networks, J. Stat. Mech. (2013) P03003.
  30. S. Sonoda and N. Murata, Transport analysis of infinitely deep neural network, arXiv:1605.02832.
  31. M. Helias and D. Dahmen, Statistical Field Theory for Neural Networks (Springer Cham, Cham, Switzerland, 2020), Vol. 970.
  32. J. Halverson, Building quantum field theories out of neurons, arXiv:2112.04527.
  33. A. Maiti, K. Stoner, and J. Halverson, Symmetry-via-duality: Invariant neural network densities from parameter-space correlators, arXiv:2106.00694.
  34. J. Halverson, J. Naskar, and J. Tian, Conformal fields from neural networks, arXiv:2409.12222.
  35. K. T. Grosvenor and R. Jefferson, The edge of chaos: Quantum field theory and deep neural networks, SciPost Phys. 12, 081 (2022).
  36. S. Gukov, J. Halverson, and F. Ruehle, Rigor with machine learning from field theory to the Poincaré conjecture, Nat. Rev. Phys. 6, 310 (2024).
  37. O. Cohen, O. Malka, and Z. Ringel, Learning curves for overparametrized deep neural networks: A field theory perspective, Phys. Rev. Res. 3, 023034 (2021).
  38. J. N. Howard, M. S. Klinger, A. Maiti, and A. G. Stapleton, Bayesian RG flow in neural network field theories, SciPost Phys. Core 8, 027 (2025).
  39. G. Aarts, D. E. Habibi, L. Wang, and K. Zhou, On learning higher-order cumulants in diffusion models, Mach. Learn. Sci. Technol. 6, 025004 (2025).
  40. M. A. Buice and J. D. Cowan, Field-theoretic approach to fluctuation effects in neural networks, Phys. Rev. E 75, 051919 (2007).
  41. D. S. Berman and M. S. Klinger, The inverse of exact renormalization group flows as statistical inference, Entropy 26, 389 (2024).
  42. D. S. Berman, M. S. Klinger, and A. G. Stapleton, Bayesian renormalization, Mach. Learn. Sci. Tech. 4, 045011 (2023).
  43. J.-W. Lee, Quantum fields as deep learning, J. Korean Phys. Soc. 76, 684 (2020).
  44. Z.-A. Jia, B. Yi, R. Zhai, Y.-C. Wu, G.-C. Guo, and G.-P. Guo, Quantum neural network states: A brief review of methods and applications, Adv. Quantum Technol. 2, 1800077 (2019).
  45. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  46. D.-L. Deng, X. Li, and S. Das Sarma, Quantum entanglement in neural network states, Phys. Rev. X 7, 021021 (2017).
  47. D. Bachtis, G. Aarts, and B. Lucini, Quantum field-theoretic machine learning, Phys. Rev. D 103, 074510 (2021).
  48. W. A. Zúñiga-Galindo, p-adic statistical field theory and deep belief networks, Physica (Amsterdam) 612A, 128492 (2023).
  49. W. A. Zúñiga-Galindo, C. He, and B. A. Zambrano-Luna, p-Adic statistical field theory and convolutional deep Boltzmann machines, Prog. Theor. Exp. Phys. 2023, 063A01 (2023).
  50. I. Banta, T. Cai, N. Craig, and Z. Zhang, Structures of neural network effective theories, Phys. Rev. D 109, 105007 (2024).
  51. V. Vanchurin, Emergent field theories from neural networks, arXiv:2411.08138.
  52. K. Segadlo, B. Epping, A. van Meegen, D. Dahmen, M. Krämer, and M. Helias, Unified field theoretical approach to deep and recurrent neuronal networks, J. Stat. Mech. (2022) 103401.
  53. R. Bondesan and M. Welling, The hintons in your neural network: A quantum field theory view of deep learning, arXiv:2103.04913.
  54. R. M. Neal, Priors for infinite networks, in Bayesian Learning for Neural Networks (Springer, New York, 1996), pp. 29–53.
  55. C. K. I. Williams, Computation with infinite neural networks, Neural Comput. 10, 1203 (1998).
  56. A. G. de G. Matthews, M. Rowland, J. Hron, R. E. Turner, and Z. Ghahramani, Gaussian process behaviour in wide deep neural networks, arXiv:1804.11271.
  57. R. Novak, L. Xiao, J. Lee, Y. Bahri, G. Yang, J. Hron, D. A. Abolafia, J. Pennington, and J. Sohl-Dickstein, Bayesian deep convolutional networks with many channels are Gaussian processes, arXiv:1810.05148.
  58. A. Garriga-Alonso, C. E. Rasmussen, and L. Aitchison, Deep convolutional networks as shallow Gaussian processes, arXiv:1808.05587.
  59. G. Yang, Scaling limits of wide neural networks with weight sharing: Gaussian process behavior, gradient independence, and neural tangent kernel derivation, arXiv:1902.04760.
  60. G. Yang, Tensor Programs I: Wide feedforward or recurrent neural networks of any architecture are Gaussian processes, arXiv:1910.12478.
  61. G. Yang, Tensor Programs II: Neural tangent kernel for any architecture, arXiv:2006.14548.
  62. A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, arXiv:1806.07572.
  63. J. Lederer, Activation functions in artificial neural networks: A systematic overview, arXiv:2101.09957.
  64. S. Ruder, An overview of gradient descent optimization algorithms, arXiv:1609.04747.
  65. J. Lee, L. Xiao, S. S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, and J. Pennington, Wide neural networks of any depth evolve as linear models under gradient descent, J. Stat. Mech. (2020) 124002.
  66. G. Parisi, Statistical Field Theory, Frontiers in Physics (Addison-Wesley, Redwood City, CA, 1988).
  67. T. L. H. Watkin, A. Rau, and M. Biehl, The statistical mechanics of learning a rule, Rev. Mod. Phys. 65, 499 (1993).
  68. M. Samarin, V. Roth, and D. Belius, On the empirical neural tangent kernel of standard finite-width convolutional neural network architectures, arXiv:2006.13645.
  69. S. Fort, G. K. Dziugaite, M. Paul, S. Kharaghani, D. M. Roy, and S. Ganguli, Deep learning versus kernel learning: An empirical study of loss landscape geometry and the time evolution of the neural tangent kernel, arXiv:2010.15110.
  70. R. Novak, L. Xiao, J. Hron, J. Lee, A. A. Alemi, J. Sohl-Dickstein, and S. S. Schoenholz, Neural tangents: Fast and easy infinite neural networks in python, arXiv:1912.02803.
  71. J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. U.S.A. 79, 2554 (1982).
  72. W. Little, The existence of persistent states in the brain, Math. Biosci. 19, 101 (1974).
  73. P. Smolensky, Information processing in dynamical systems: Foundations of harmony theory, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1—Foundations, edited by D. E. Rumelhart and J. L. McClelland (MIT Press, Cambridge, MA, 1986), pp. 194–281.
  74. D. Lee, H.-S. Lee, and J. Yi, Dynamic neuron approach to deep neural networks: Decoupling neurons for renormalization group analysis, Phys. Rev. Res. 7, 023276 (2025).

Outline

Information

Phys. Rev. D 112, L031902– Published 15 August, 2025
  • Received 25 March 2025
  • Accepted 25 July 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation