- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Temperature steerable flows and Boltzmann generators

Manuel Dibak1,*, Leon Klein1,*, Andreas Krämer1, and Frank Noé1,2,3,4,†

  • 1Department of Mathematics and Computer Science, Freie Universität Berlin, Arnimallee 12, 14195 Berlin, Germany
  • 2Department of Physics, Freie Universität Berlin, Arnimallee 12, 14195 Berlin, Germany
  • 3Department of Chemistry, Rice University, Houston, Texas 77005, USA
  • 4Microsoft Research, Station Road, Cambridge, United Kingdom

  • *These authors contributed equally to this work.
  • †Corresponding author: frank.noe@fu-berlin.de
PDF

Phys. Rev. Research 4, L042005 – Published 10 October, 2022

DOI: https://doi.org/10.1103/PhysRevResearch.4.L042005

Abstract

Boltzmann generators approach the sampling problem in many-body physics by combining a normalizing flow and a statistical reweighting method to generate samples in thermodynamic equilibrium. The equilibrium distribution is usually defined by an energy function and a thermodynamic state. Here, we propose temperature steerable flows (TSFs) which are able to generate a family of probability densities parametrized by a choosable temperature parameter. TSFs can be embedded in generalized ensemble sampling frameworks to sample a physical system across multiple thermodynamic states.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (37)

  1. M. Böttcher, S. Heinze, S. Egorov, J. Sinova, and B. Dupé, B−T phase diagram of Pd/Fe/Ir(111) computed with parallel tempering Monte Carlo, New J. Phys. 20, 103014 (2018).
  2. R. Wuttke, H. Hofmann, D. Nettels, M. B. Borgia, J. Mittal, R. B. Best, and B. Schuler, Temperature-dependent solvation modulates the dimensions of disordered proteins, Proc. Natl. Acad. Sci. U.S.A. 111, 5213 (2014).
  3. D. J. Earl and M. W. Deem, Parallel tempering: Theory, applications, and new perspectives, Phys. Chem. Chem. Phys. 7, 3910 (2005).
  4. R. H. Swendsen and J.-S. Wang, Replica Monte Carlo Simulation of Spin-Glasses, Phys. Rev. Lett. 57, 2607 (1986).
  5. I. Kobyzev, S. Prince, and M. Brubaker, Normalizing flows: An introduction and review of current methods, IEEE Trans. Pattern Anal. Mach. Intell. 43, 3964 (2021).
  6. S.-H. Li, C.-X. Dong, L. Zhang, and L. Wang, Neural Canonical Transformation with Symplectic Flows, Phys. Rev. X 10, 021020 (2020).
  7. G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, Normalizing flows for probabilistic modeling and inference, J. Mach. Learn. Res. 22, 1 (2021).
  8. D. Rezende and S. Mohamed, Variational inference with normalizing flows, in Proceedings of the 32nd International Conference on Machine Learning, edited by F. Bach and D. Blei, Proceedings of Machine Learning Research Vol. 37 (PMLR, 2015), pp. 1530–1538.
  9. E. G. Tabak and E. Vanden-Eijnden, Density estimation by dual ascent of the log-likelihood, Commun. Math. Sci. 8, 217 (2010).
  10. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.4.L042005 for a more detailed introduction of normalizing flows, more experiments, as well as additional information.
  11. F. Noé, S. Olsson, J. Köhler, and H. Wu, Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning, Science 365, eaaw1147 (2019).
  12. H. Wu, J. Köhler, and F. Noé, Stochastic normalizing flows, Adv. Neural Inf. Process. Syst. 33, 5933 (2020).
  13. K. A. Nicoli, C. J. Anders, L. Funcke, T. Hartung, K. Jansen, P. Kessel, S. Nakajima, and P. Stornati, Estimation of Thermodynamic Observables in Lattice Field Theories with Deep Generative Models, Phys. Rev. Lett. 126, 032001 (2021).
  14. S. Li and L. Wang, Neural Network Renormalization Group, Phys. Rev. Lett. 121, 260601 (2018).
  15. K. A. Nicoli, S. Nakajima, N. Strodthoff, W. Samek, K. Müller, and P. Kessel, Asymptotically unbiased estimation of physical observables with neural samplers, Phys. Rev. E 101, 023304 (2020).
  16. M. S. Albergo, G. Kanwar, S. Racanière, D. J. Rezende, J. M. Urban, D. Boyda, K. Cranmer, D. C. Hackett, and P. E. Shanahan, Flow-based sampling for fermionic lattice field theories, Phys. Rev. D 104, 114507 (2021).
  17. M. S. Albergo, G. Kanwar, and P. E. Shanahan, Flow-based generative models for Markov chain Monte Carlo in lattice field theory, Phys. Rev. D 100, 034515 (2019).
  18. D. Boyda, G. Kanwar, S. Racanière, D. J. Rezende, M. S. Albergo, K. Cranmer, D. C. Hackett, and P. E. Shanahan, Sampling using SU(N) gauge equivariant flows, Phys. Rev. D 103, 074504 (2021).
  19. G. Kanwar, M. S. Albergo, D. Boyda, K. Cranmer, D. C. Hackett, S. Racaniere, D. J. Rezende, and P. E. Shanahan, Equivariant Flow-Based Sampling for Lattice Gauge Theory, Phys. Rev. Lett. 125, 121601 (2020).
  20. H. E. Stanley, Dependence of Critical Properties on Dimensionality of Spins, Phys. Rev. Lett. 20, 589 (1968).
  21. C. Huang, L. Dinh, and A. C. Courville, Augmented normalizing flows: Bridging the gap between generative flows and latent variable models, arXiv:2002.07101.
  22. S. Duane, A. D. Kennedy, B. J. Pendleton, and D. Roweth, Hybrid Monte Carlo, Phys. Lett. B 195, 216 (1987).
  23. S. Greydanus, M. Dzamba, and J. Yosinski, Hamiltonian neural networks, Adv. Neural Inf. Process. Syst. 32, 15379 (2019).
  24. P. Toth, Danilo Jimenez Rezende, A. Jaegle, S. Racanière, A. Botev, and I. Higgins, Hamiltonian generative networks, arXiv:1909.13789.
  25. L. Dinh, J. Sohl-Dickstein, and S. Bengio, Density estimation using real NVP, arXiv:1605.08803.
  26. P. Sorrenson, C. Rother, and U. Köthe, Disentanglement by nonlinear ICA with general incompressible-flow networks (GIN), arXiv:2001.04872.
  27. C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios, Neural spline flows, Adv. Neural Inf. Process. Syst. 32, 32 (2019).
  28. D. J. Rezende, G. Papamakarios, S. Racaniere, M. Albergo, G. Kanwar, P. Shanahan, and K. Cranmer, Normalizing flows on tori and spheres, in Proceedings of the 37th International Conference on Machine Learning, edited by H. Daumé III and A. Singh, Proceedings of Machine Learning Research, Vol. 119 (PMLR, 2020), pp. 8083–8092.
  29. L. Dinh, D. Krueger, and Y. Bengio, NICE: Nonlinear independent components estimation, arXiv:1410.8516.
  30. C. J. Geyer, Markov chain Monte Carlo maximum likelihood, in Proceedings of the 23rd Symposium on the Interface, edited by E. M. Keramidas (Interface Foundation of North America, Fairfax Station, VA, 1991), pp. 156–163.
  31. K. Hukushima and K. Nemoto, Exchange Monte Carlo method and application to spin glass simulations, J. Phys. Soc. Jpn. 65, 1604 (1996).
  32. R. J. Glauber, Time-dependent statistics of the Ising model, J. Math. Phys. 4, 294 (1963).
  33. G. Pérez-Hernández, F. Paul, T. Giorgino, G. De Fabritiis, and F. Noé, Identification of slow molecular order parameters for Markov model construction, J. Chem. Phys. 139, 015102 (2013).
  34. J. Köhler, L. Klein, and F. Noé, Equivariant flows: exact likelihood generative learning for symmetric densities, in Proceedings of the 37th International Conference on Machine Learning (Ref. [28]), pp. 5361–5370.
  35. D. J. Rezende, S. Racanière, I. Higgins, and P. Toth, Equivariant Hamiltonian flows, arXiv:1909.13739.
  36. V. G. Satorras, E. Hoogeboom, F. Fuchs, I. Posner, and M. Welling, E(n) equivariant normalizing flows, in Advances in Neural Information Processing Systems, edited by M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan (Curran Associates, Inc., 2021) Vol. 34, pp. 4181–4192.
  37. L. Zhang, Weinan E, and L. Wang, Monge-Ampère flow for generative modeling, arXiv:1809.10188.

Outline

Information

Phys. Rev. Research 4, L042005– Published 10 October, 2022
  • Received 4 August 2021
  • Accepted 10 August 2022
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation