- Physical Review Journals
All JournalsPhysics Magazine

Physical Review E

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Universal spin models are universal approximators in machine learning

Tobias Reinhart*

Gemma De les Coves

  • *Contact author: tobias.reinhart@uibk.ac.at
PDF

Phys. Rev. E 113, L043301 – Published 14 April, 2026

DOI: https://doi.org/10.1103/b3xp-wtwx

Abstract

One of the theoretical pillars that sustain certain machine learning models are universal approximation theorems, which prove that they can approximate all functions from a function class to arbitrary precision. Independently, classical spin models are termed universal if they can reproduce the behavior of any other spin model in their low energy sector. Universal spin models have been characterized via sufficient and necessary conditions, showing that simple models such as the 2d Ising with fields are universal. In this work, we prove that universal spin models are universal approximators of probability distributions. This enables us to leverage the characterization of the former to reveal conditions which are sufficient for universal approximation. Deriving universal approximation theorems thus amounts to verifying these conditions, yielding a unified recipe for universal approximation theorems applicable to a wide range of models. We explicitly test this recipe for restricted and deep Boltzmann machines, as well as for deep belief networks. This work illustrates that independently discovered universality statements may be intimately related, enabling the transfer of results.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (29)

  1. E. Ising, Beitrag zur Theorie des Ferromagnetismus, Z. Angew. Phys. 31, 253 (1925).
  2. T. Reinhart and G. De les Coves, The grammar of the Ising model: A new complexity hierarchy, Proc. R. Soc. A 481, 20240579 (2025).
  3. T. Osogami, Boltzmann machines and energy-based models, arXiv:1708.06008.
  4. N. Le Roux and Y. Bengio, Representational power of restricted Boltzmann machines and deep belief networks, Neural Comput. 20, 1631 (2008).
  5. N. Le Roux and Y. Bengio, Deep belief networks are compact universal approximators, Neural Comput. 22, 2192 (2010).
  6. G. Montúfar, Deep narrow Boltzmann machines are universal approximators, in Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, arXiv:1411.3784 (2015).
  7. G. Montúfar and J. Rauh, Hierarchical models as marginals of hierarchical models, Int. J. Approximate Reasoning 88, 531 (2017).
  8. G. De las Cuevas and T. S. Cubitt, Simple universal models capture all classical spin physics, Science 351, 1180 (2016).
  9. T. Reinhart, B. Engel, and G. D. les Coves, The structure of emulations in classical spin models: Modularity and universality, arXiv:2407.13428.
  10. That is machine learning models, defined by a Hamiltonian the corresponding Boltzmann distribution, examples are DBMs and RBMs.
  11. See Supplemental Material at http://link.aps.org/supplemental/10.1103/b3xp-wtwx for additional details on simulations.
  12. Simulations can be composed, scaled and added [9]. This implies, e.g. that they can be constructed from gadgets, spin systems that locally replace a part of the target, similar to Eq.  (2).
  13. We denote by 0 the all zero and by 1 the all one vector.
  14. I. Sutskever and G. E. Hinton, Deep, narrow sigmoid belief networks are universal approximators, Neural Comput. 20, 2629 (2008).
  15. G. Hinton, Training products of experts by minimizing contrastive divergence, Neural Comput. 14, 1771 (2002).
  16. Y. Song and D. P. Kingma, How to train your energy-based models, arXiv:2101.03288.
  17. D. Carbone, Hitchhiker's guide on the relation of energy-based models with other generative models, sampling and statistical physics: A comprehensive review, Transactions on Machine Learning Research (2025).
  18. X. Gao and L.-M. Duan, Efficient representation of quantum many-body states with deep neural networks, Nat. Commun. 8, 662 (2017).
  19. G. Carleo, Y. Nomura, and M. Imada, Constructing exact representations of quantum many-body systems with deep neural networks, Nat. Commun. 9, 5322 (2018).
  20. L. Gu, F. Zhou, and L. Yang, Towards the representational power of restricted Boltzmann machines, Neurocomputing 415, 358 (2020).
  21. J. Martens, A. Chattopadhya, T. Pitassi, and R. Zemel, On the representational efficiency of restricted Boltzmann machines, in Advances in Neural Information Processing Systems, Vol. 26 (2013).
  22. K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2, 359 (1989).
  23. G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signal Systems 2, 303 (1989).
  24. R. Sato, A survey on the expressive power of graph neural networks, arXiv:2003.04078.
  25. C. Yun, S. Bhojanapalli, A. S. Rawat, S. Reddi, and S. Kumar, Are transformers universal approximators of sequence-to-sequence functions? in Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia (2020).
  26. T. Gonda, T. Reinhart, S. Stengele, and G. D. les Coves, A framework for universality in physics, computer science, and beyond, Compositionality 6, 3 (2024).
  27. H. Siegelmann and E. Sontag, On the computational power of neural nets, J. Comput. Syst. Sci. 50, 132 (1995).
  28. J. Pérez, J. Marinković, and P. Barceló, On the Turing completeness of modern neural network architectures, in Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, Louisiana, USA (2019).
  29. P. Fraser, R. Solé, and G. De las Cuevas, Why can the brain (and not a computer) make sense of the liar paradox? Front. Ecol. Evol. 9, 802300 (2021).

Outline

Information

Phys. Rev. E 113, L043301– Published 14 April, 2026
  • Received 11 July 2025
  • Accepted 9 March 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation