- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Learning Hamiltonian neural Koopman operator and simultaneously sustaining and discovering conservation laws

Jingdong Zhang1,2, Qunxi Zhu2,3,4,*, and Wei Lin1,2,3,4,†

  • 1School of Mathematical Sciences, SCMS, and SCAM, Fudan University, Shanghai 200433, China
  • 2Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China
  • 3Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China
  • 4MOE Frontiers Center for Brain Science and State Key Laboratory of Medical Neurobiology, Fudan University, Shanghai 200032, China

  • *qxzhu16@fudan.edu.cn
  • †wlin@fudan.edu.cn
PDF

Phys. Rev. Research 6, L012031 – Published 20 February, 2024

DOI: https://doi.org/10.1103/PhysRevResearch.6.L012031

Abstract

Accurately finding and predicting dynamics based on the observational data with noise perturbations is of paramount significance but still a major challenge presently. Here, for the Hamiltonian mechanics, we propose the Hamiltonian neural Koopman operator (HNKO), integrating the knowledge of mathematical physics in learning the Koopman operator, and making it automatically sustain and even discover the conservation laws. We demonstrate the outperformance of the HNKO and its extension using a number of representative physical systems even with hundreds or thousands of freedoms. Our results suggest that feeding the prior knowledge of the underlying system and the mathematical theory appropriately to the learning framework can reinforce the capability of machine learning in solving physical problems.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (61)

  1. S. Greydanus, M. Dzamba, and J. Yosinski, Hamiltonian neural networks, in Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada (Curran Associates, Inc., Vancouver, 2019), Vol. 32.
  2. C.-D. Han, B. Glaz, M. Haile, and Y.-C. Lai, Adaptable Hamiltonian neural networks, Phys. Rev. Res. 3, 023156 (2021).
  3. M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, and S. Ho, Lagrangian neural networks, in ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations (OpenReview, Addis Ababa, 2019).
  4. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, Neural ordinary differential equations, in Advances in Neural Information Processing Systems (Curran Associates, New York, 2019), Vol. 31.
  5. Q. Zhu, Y. Guo, and W. Lin, Neural delay differential equations, in International Conference on Learning Representations (OpenReview, Virtual, 2021).
  6. Q. Zhu, Y. Shen, D. Li, and W. Lin, Neural piecewise-constant delay differential equations, in Proceedings of the AAAI Conference on Artificial Intelligence (OpenReview, Virtual, 2022), Vol. 36, pp. 9242–9250.
  7. M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378, 686 (2019).
  8. G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nature Rev. Phys. 3, 422 (2021).
  9. F. de Avila Belbute-Peres, Y.-f. Chen, and F. Sha, HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks, in The Symbiosis of Deep Learning and Differential Equations (OpenReview, Sydney, 2021).
  10. L. Lu, R. Pestourie, W. Yao, Z. Wang, F. Verdugo, and S. G. Johnson, Physics-informed neural networks with hard constraints for inverse design, SIAM J. Sci. Comput. 43, B1105 (2021).
  11. W. Chen, Q. Wang, J. S. Hesthaven, and C. Zhang, Physics-informed machine learning for reduced-order modeling of nonlinear problems, J. Comput. Phys. 446, 110666 (2021).
  12. H. Eivazi, M. Tahani, P. Schlatter, and R. Vinuesa, Physics-informed neural networks for solving reynolds-averaged navier–stokes equations, Phys. Fluids 34, 075117 (2022).
  13. S. Cai, Z. Mao, Z. Wang, M. Yin, and G. E. Karniadakis, Physics-informed neural networks (PINNs) for fluid mechanics: A review, Acta Mech. Sin. 37, 1727 (2021).
  14. Z. Mao, A. D. Jagtap, and G. E. Karniadakis, Physics-informed neural networks for high-speed flows, Comput. Methods Appl. Mech. Eng. 360, 112789 (2020).
  15. G. S. Misyris, A. Venzke, and S. Chatzivasileiadis, Physics-informed neural networks for power systems, in IEEE PES GM (IEEE, New York, 2020), pp. 1–5.
  16. Q. Zhu, H. Ma, and W. Lin, Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing, Chaos 29, 093125 (2019).
  17. Y. Tang, J. Kurths, W. Lin, E. Ott, and L. Kocarev, When machine learning meets complex systems: Networks, chaos, and nonlinear dynamics, Chaos 30, 063151 (2020).
  18. X.-Y. Duan, X. Ying, S.-Y. Leng, J. Kurths, W. Lin, and H.-F. Ma, Embedding theory of reservoir computing and reducing reservoir network using time delays, Phys. Rev. Res. 5, L022041 (2023).
  19. P. Toth, D. J. Rezende, A. Jaegle, S. Racanière, A. Botev, and I. Higgins, Hamiltonian generative networks, in International Conference on Learning Representations (OpenReview, Addis Ababa, 2020).
  20. Y. D. Zhong, B. Dey, and A. Chakraborty, Symplectic ODE-Net: Learning Hamiltonian dynamics with control, in International Conference on Learning Representations (OpenReview, Addis Ababa, 2020).
  21. A. Sanchez-Gonzalez, V. Bapst, K. Cranmer, and P. Battaglia, Hamiltonian graph networks with ode integrators, in NeurIPS Workshop on Machine Learning and the Physical Sciences (OpenReview, Vancouver, 2019).
  22. H. Ma, W. Lin, and Y.-C. Lai, Detecting unstable periodic orbits in high-dimensional chaotic systems from time series: Reconstruction meeting with adaptation, Phys. Rev. E 87, 050901(R) (2013).
  23. Q. Zhu, X. Li, and W. Lin, Leveraging neural differential equations and adaptive delayed feedback to detect unstable periodic orbits based on irregularly sampled time series, Chaos 33, 031101 (2023).
  24. J. Zhang, Q. Zhu, and W. Lin, Neural stochastic control, in Advances in Neural Information Processing Systems (Curran Associates, Inc., New Orleans, 2022), Vol. 35, pp. 9098–9110.
  25. J. Zhang, Q. Zhu, W. Yang, and W. Lin, Sync: Safety-aware neural control for stabilizing stochastic delay-differential equations, in International Conference on Learning Representations (OpenReview, Kigali, 2023).
  26. D. Rolnick, P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. S. Ross, N. Milojevic-Dupont, N. Jaques, A. Waldman-Brown et al., Tackling climate change with machine learning, ACM Comput. Surv. 55, 1 (2023).
  27. X. Li, Q. Zhu, C. Zhao, X. Qian, X. Zhang, X. Duan, and W. Lin, Tipping point detection using reservoir computing, Research 6, 0174 (2023).
  28. P. Zhang, H. Shen, and H. Zhai, Machine learning topological invariants with neural networks, Phys. Rev. Lett. 120, 066401 (2018).
  29. Z. Liu and M. Tegmark, Machine learning conservation laws from trajectories, Phys. Rev. Lett. 126, 180604 (2021).
  30. M. Lezcano-Casado and D. Martınez-Rubio, Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group, in Proceedings of the 36th International Conference on Machine Learning (PMLR, Long Beach, 2019), Vol. 97.
  31. M. Lezcano-Casado, Trivializations for gradient-based optimization on manifolds, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Vancouver, 2019), Vol. 32.
  32. S. L. Brunton, M. Budisic, E. Kaiser, and J. N. Kutz, Modern Koopman theory for dynamical systems, SIAM Rev. 64, 229 (2022).
  33. I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dyn. 41, 309 (2005).
  34. C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech. 641, 115 (2009).
  35. B. Lusch, J. N. Kutz, and S. L. Brunton, Deep learning for universal linear embeddings of nonlinear dynamics, Nature Commun. 9, 4950 (2018).
  36. S. L. Brunton, B. R. Noack, and P. Koumoutsakos, Machine learning for fluid mechanics, Annu. Rev. Fluid Mech. 52, 477 (2020).
  37. A. Otto, D. Müller, and G. Radons, Universal dichotomy for dynamical systems with variable delay, Phys. Rev. Lett. 118, 044104 (2017).
  38. P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech. 656, 5 (2010).
  39. J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, On dynamic mode decomposition: Theory and applications, J. Comput. Dynam. 1, 391 (2014).
  40. M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, A data–driven approximation of the koopman operator: Extending dynamic mode decomposition, J. Nonlinear Sci. 25, 1307 (2015).
  41. S. L. Brunton and J. N. Kutz, Data-Driven Science and Engineering Machine Learning, Dynamical Systems, and Control (Cambridge University Press, Cambridge, 2022).
  42. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.6.L012031 for the theories of the manifold dimension and the orthogonal projection of the Koopman operator, the reduced computational cost, the discovered Hamiltonian, the mechanism of filtering the noise, the illustrations on the simulation configurations, and the limitations and perspectives.
  43. C. F. Loan, The ubiquitous kronecker product, J. Comput. Appl. Math. 123, 85 (2000).
  44. M. Valtonen and H. Karttunen, The Three-Body Problem (Cambridge University Press, Cambridge, 2006).
  45. P. Jin, Z. Zhang, A. Zhu, Y. Tang, and G. E. Karniadakis, Sympnets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems, Neural Networks 132, 166 (2020).
  46. R. Mutegeki and D. S. Han, A cnn-lstm approach to human activity recognition, in 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (IEEE, New York, 2020), pp. 362–366.
  47. A. Vidal and W. Kristjanpoller, Gold volatility prediction using a cnn-lstm approach, Exp. Syst. Appl. 157, 113481 (2020).
  48. J. Pathak, B. Hunt, M. Girvan, Z. Lu, and E. Ott, Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach, Phys. Rev. Lett. 120, 024102 (2018).
  49. H. Zhang, H. Fan, L. Wang, and X. Wang, Learning Hamiltonian dynamics with reservoir computing, Phys. Rev. E 104, 024205 (2021).
  50. V. I. Arnold, Mathematical Methods of Classical Mechanics (Springer, New York, 2013), Vol. 60.
  51. G. Fusco, G. F. Gronchi, and P. Negrini, Platonic polyhedra, topological constraints and periodic solutions of the classical n-body problem, Invent. Math. 185, 283 (2011).
  52. G. R. Fowles, G. L. Cassiday et al., Analytical Mechanics (Saunders College, Rochester, 1999).
  53. S. Kim, W. Ji, S. Deng, Y. Ma, and C. Rackauckas, Stiff neural ordinary differential equations, Chaos 31, 093122 (2021).
  54. A. Ghosh, H. Behl, E. Dupont, P. Torr, and V. Namboodiri, STEER: Simple temporal regularization for neural ode, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Vancouver, 2020), Vol. 33.
  55. R. M. Miura, C. S. Gardner, and M. D. Kruskal, Korteweg-de vries equation and generalizations. II. Existence of conservation laws and constants of motion, J. Math. Phys. 9, 1204 (1968).
  56. N. Zabusky and C. Galvin, Shallow-water waves, the korteweg-devries equation and solitons, J. Fluid Mech. 47, 811 (1971).
  57. E. Kaiser, J. N. Kutz, and S. L. Brunton, Discovering conservation laws from data for control, in IEEE Conference on Decision and Control (IEEE, New York, 2018), pp. 6415–6421.
  58. S. Liang, Z. Huang, and H. Zhang, Stiffness-aware neural network for learning Hamiltonian systems, in International Conference on Learning Representations (OpenReview, Virtual, 2021).
  59. Z. Chen, J. Zhang, M. Arjovsky, and L. Bottou, Symplectic recurrent neural networks, in International Conference on Learning Representations (OpenReview, New Orleans, 2019).
  60. P. Y. Lu, R. Dangovski, and M. Soljačić, Discovering conservation laws using optimal transport and manifold learning, Nature Commun. 14, 4744 (2023).
  61. R. Iten, T. Metger, H. Wilming, L. del Rio, and R. Renner, Discovering physical concepts with neural networks, Phys. Rev. Lett. 124, 010508 (2020).

Outline

Information

Phys. Rev. Research 6, L012031– Published 20 February, 2024
  • Received 7 March 2023
  • Revised 21 September 2023
  • Accepted 24 January 2024
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation