- Physical Review Journals
All JournalsPhysics Magazine

Physical Review E

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Ratio divergence learning using target energy in restricted Boltzmann machines: Beyond Kullback-Leibler divergence learning

Yuichi Ishida1,*, Yuma Ichikawa1,2,†, Aki Dote1,‡, Toshiyuki Miyazawa1,§, and Koji Hukushima2,∥

  • *Contact author: ishida-yuichi@fujitsu.com
  • †Contact author: ichikawa.yuma@fujitsu.com
  • ‡Contact author: dote.aki@fujitsu.com
  • §Contact author: miyazawa.toshi@fujitsu.com
  • ∥Contact author: k-hukushima@g.ecc.u-tokyo.ac.jp
PDF

Phys. Rev. E 112, 045306 – Published 7 October, 2025

DOI: https://doi.org/10.1103/fxnm-y5pd

Abstract

We propose ratio divergence (RD) learning for discrete energy-based models, a method that utilizes both training data and a tractable target energy function. We apply RD learning to restricted Boltzmann machines (RBMs), which are a minimal model that satisfies the universal approximation theorem for discrete distributions. RD learning combines the strength of both forward and reverse Kullback–Leibler divergence (KLD) learning, effectively addressing the “notorious” issues of underfitting with the forward KLD and mode collapse with the reverse KLD. Since the summation of forward and reverse KLD seems to be sufficient to combine the strength of both approaches, we include this learning method as a direct baseline in numerical experiments to evaluate its effectiveness. Numerical experiments demonstrate that RD learning outperforms other learning methods in terms of energy function fitting, mode-covering, and learning stability across various discrete energy-based models. Moreover, the performance gaps between RD learning and the other learning methods become more pronounced as the dimensions of target models increase.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (41)

  1. A. Baumgärtner, A. N. Burkitt, D. M. Ceperley, H. De Raedt, A. M. Ferrenberg, D. W. Heermann, H. J. Herrmann, D. P. Landau, D. Levesque, W. von der Linden et al., The Monte Carlo Method in Condensed Matter Physics (Springer Science & Business Media, New York, 2012), Vol. 71.
  2. K. Binder, D. Heermann, L. Roelofs, A. J. Mallinckrodt, and S. McKay, Monte Carlo simulation in statistical physics, Comput. Phys. 7, 156 (1993).
  3. D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman, emcee: the MCMC hammer, Publ. Astron. Soc. Pac. 125, 306 (2013).
  4. A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis (Chapman and Hall/CRC Press, Boca Raton, Florida, 1995).
  5. A. D. Martin, K. M. Quinn, and J. H. Park, MCMCpack: Markov chain Monte Carlo in R, J. Stat. Softw. 42, 1 (2011).
  6. K. Dabiri, M. Malekmohammadi, A. Sheikholeslami, and H. Tamura, Replica exchange MCMC hardware with automatic temperature selection and parallel trial, IEEE Trans. Parallel Distrib. Syst. 31, 1681 (2020).
  7. J. Gu and K. Zhang, Thermodynamics of the Ising model encoded in restricted Boltzmann machines, Entropy 24, 1701 (2022).
  8. W. Chen, A. R. Tan, and A. L. Ferguson, Collective variable discovery and enhanced sampling using autoencoders: Innovations in network architecture and error function design, J. Chem. Phys. 149, 072312 (2018).
  9. J. M. L. Ribeiro, P. Bravo, Y. Wang, and P. Tiwary, Reweighted autoencoded variational Bayes for enhanced sampling (RAVE), J. Chem. Phys. 149, 072301 (2018).
  10. L. Huang and L. Wang, Accelerated Monte Carlo simulations with restricted Boltzmann machines, Phys. Rev. B 95, 035105 (2017).
  11. J. Liu, Y. Qi, Z. Y. Meng, and L. Fu, Self-learning Monte Carlo method, Phys. Rev. B 95, 041101(R) (2017).
  12. X. Y. Xu, Y. Qi, J. Liu, L. Fu, and Z. Y. Meng, Self-learning quantum Monte Carlo method in interacting fermion systems, Phys. Rev. B 96, 041119(R) (2017).
  13. V. Stimper, B. Schölkopf, and J. M. Hernández-Lobato, Resampling base distributions of normalizing flows, in International Conference on Artificial Intelligence and Statistics (PMLR, 2022), pp. 4915–4936.
  14. P. Wirnsberger, G. Papamakarios, B. Ibarz, S. Racanière, A. J. Ballard, A. Pritzel, and C. Blundell, Normalizing flows for atomic solids, Mach. Learn.: Sci. Technol. 3, 025009 (2022).
  15. H. Wu, J. Köhler, and F. Noé, Stochastic normalizing flows, in Advances in Neural Information Processing Systems, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Curran Associates, Inc., 2020), Vol. 33, pp. 5933–5944.
  16. S. Ciarella, J. Trinquier, M. Weigt, and F. Zamponi, Machine-learning-assisted Monte Carlo fails at sampling computationally hard problems, Mach. Learn.: Sci. Technol. 4, 010501 (2023).
  17. G. E. Hinton, Training products of experts by minimizing contrastive divergence, Neural Comput. 14, 1771 (2002).
  18. D. E. Rumelhart, J. L. McClelland, and PDP Research Group, Information processing in dynamical systems: Foundations of Harmony theory, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations (MIT Press, Cambridge, Massachusetts, 1986), Vol. 1, pp.194–281.
  19. N. Le Roux and Y. Bengio, Representational power of restricted Boltzmann machines and deep belief networks, Neural Comput. 20, 1631 (2008).
  20. R. D. Hjelm, V. D. Calhoun, R. Salakhutdinov, E. A. Allen, T. Adali, and S. M. Plis, Restricted Boltzmann machines for neuroimaging: an application in identifying intrinsic networks, NeuroImage 96, 245 (2014).
  21. R. G. Melko, G. Carleo, J. Carrasquilla, and J. I. Cirac, Restricted Boltzmann machines in quantum physics, Nat. Phys. 15, 887 (2019).
  22. A. Khajenezhad, H. Madani, and H. Beigy, Masked autoencoder for distribution estimation on small structured data sets, IEEE Trans. Neural Netw. Learn. Syst. 32, 4997 (2020).
  23. B. Uria, M.-A. Côté, K. Gregor, I. Murray, and H. Larochelle, Neural autoregressive distribution estimation, J. Mach. Learn. Res. 17, 1 (2016).
  24. T. Tieleman, Training restricted Boltzmann machines using approximations to the likelihood gradient, in Proceedings of the 25th International Conference on Machine Learning ICML '08, (Association for Computing Machinery, New York, NY, USA, 2008), pp. 1064–1071.
  25. D. A. Puente and I. M. Eremin, Convolutional restricted Boltzmann machine aided Monte Carlo: an application to Ising and Kitaev models, Phys. Rev. B 102, 195148 (2020).
  26. B. McNaughton, M. V. Milošević, A. Perali, and S. Pilati, Boosting Monte Carlo simulations of spin glasses using autoregressive neural networks, Phys. Rev. E 101, 053312 (2020).
  27. D. Wu, R. Rossi, and G. Carleo, Unbiased Monte Carlo cluster updates with autoregressive neural networks, Phys. Rev. Res. 3, L042024 (2021).
  28. D. Wu, L. Wang, and P. Zhang, Solving statistical mechanics using variational autoregressive networks, Phys. Rev. Lett. 122, 080602 (2019).
  29. C. Fan, M. Shen, Z. Nussinov, Z. Liu, Y. Sun, and Y.-Y. Liu, Finding spin glass ground states through deep reinforcement learning, arXiv:2109.14411.
  30. M. Hibat-Allah, E. M. Inack, R. Wiersema, R. G. Melko, and J. Carrasquilla, Variational neural annealing, Nat Mach Intell 3, 952 (2021).
  31. E. M. Inack, S. Morawetz, and R. G. Melko, Neural annealing and visualization of autoregressive neural networks in the Newman–Moore model, Condens. Matter 7, 38 (2022).
  32. A. Hyvärinen, Some extensions of score matching, Comput. Stat. Data Anal. 51, 2499 (2007).
  33. L. I. Midgley, V. Stimper, G. N. C. Simm, B. Schölkopf, and J. M. Hernández-Lobato, Flow annealed importance sampling bootstrap, arXiv:2208.01893.
  34. H. Fernau, P. Golovach, M.-F. Sagot et al., Algorithmic enumeration: Output-sensitive, input-sensitive, parameterized, approximative (Dagstuhl seminar 18421), in Dagstuhl Reports (Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 2019), Vol. 8, pp. 63–86.
  35. T. Hanaka, M. Kiyomi, Y. Kobayashi, Y. Kobayashi, K. Kurita, and Y. Otachi, A framework to design approximation algorithms for finding diverse solutions in combinatorial problems, Proc. AAAI Conf. Artif. Intell. 37, 3968 (2023).
  36. K. Hukushima and K. Nemoto, Exchange Monte Carlo method and application to spin glass simulations, J. Phys. Soc. Jpn. 65, 1604 (1996).
  37. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980.
  38. Y. Ye, Gset (2003), http://web.stanford.edu/∼yyye/yyye/Gset/.
  39. N. Béreux, A. Decelle, C. Furtlehner, L. Rosset, and B. Seoane, Fast, accurate training and sampling of restricted Boltzmann machines, arXiv:2405.15376.
  40. A. Decelle, C. Furtlehner, and B. Seoane, Equilibrium and non-equilibrium regimes in the learning of restricted Boltzmann machines, in Proceedings of the 35th International Conference on Neural Information Processing System (Curran Associates Inc., Red Hook, NY, 2021), pp. 5345–5359.
  41. E. Nijkamp, M. Hill, S.-C. Zhu, and Y. N. Wu, Learning non-convergent non-persistent short-run MCMC toward energy-based model, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Red Hook, New York, 2019), Vol. 32.

Outline

Information

Phys. Rev. E 112, 045306– Published 7 October, 2025
  • Received 9 October 2024
  • Accepted 24 July 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation