- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Discovering invariants via machine learning

Seungwoong Ha and Hawoong Jeong*

  • Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea

  • *Also at Center for Complex Systems, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea; hjeong@kaist.edu
PDF

Phys. Rev. Research 3, L042035 – Published 9 December, 2021

DOI: https://doi.org/10.1103/PhysRevResearch.3.L042035

Abstract

Invariants and conservation laws convey critical information about the underlying dynamics of a system, yet it is generally infeasible to find them from large-scale data without any prior knowledge or human insight. We propose ConservNet to achieve this goal, a neural network that spontaneously discovers a conserved quantity from grouped data where the members of each group share invariants, similar to a general experimental setting where trajectories from different trials are observed. As a neural network trained with an intuitive loss function called noise-variance loss, ConservNet learns the hidden invariants in each group of multidimensional observables in a data-driven end-to-end manner. Our model successfully discovers underlying invariants from the simulated systems having invariants as well as a real-world double-pendulum trajectory. Since the model is robust to various noises and data conditions compared to the baseline, our approach is directly applicable to experimental data for discovering hidden conservation laws and further, general relationships between variables.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (43)

  1. J. Carrasquilla and R. G. Melko, Machine learning phases of matter, Nat. Phys. 13, 431 (2017).
  2. K. Ch'ng, J. Carrasquilla, R. G. Melko, and E. Khatami, Machine Learning Phases of Strongly Correlated Fermions, Phys. Rev. X 7, 031038 (2017).
  3. E. P. Van Nieuwenburg, Y.-H. Liu, and S. D. Huber, Learning phase transitions by confusion, Nat. Phys. 13, 435 (2017).
  4. Y. Zhang and E.-A. Kim, Quantum Loop Topography for Machine Learning, Phys. Rev. Lett. 118, 216401 (2017).
  5. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  6. P. Baldi, P. Sadowski, and D. Whiteson, Searching for exotic particles in high-energy physics with deep learning, Nat. Commun. 5, 4308 (2014).
  7. P. Ponte and R. G. Melko, Kernel methods for interpretable machine learning of order parameters, Phys. Rev. B 96, 205146 (2017).
  8. P. Zhang, H. Shen, and H. Zhai, Machine Learning Topological Invariants with Neural Networks, Phys. Rev. Lett. 120, 066401 (2018).
  9. N. Sun, J. Yi, P. Zhang, H. Shen, and H. Zhai, Deep learning topological invariants of band insulators, Phys. Rev. B 98, 085402 (2018).
  10. G. Torlai, G. Mazzola, J. Carrasquilla, M. Troyer, R. Melko, and G. Carleo, Neural-network quantum state tomography, Nat. Phys. 14, 447 (2018).
  11. M. Rafayelyan, J. Dong, Y. Tan, F. Krzakala, and S. Gigan, Large-Scale Optical Reservoir Computing for Spatiotemporal Chaotic Systems Prediction, Phys. Rev. X 10, 041037 (2020).
  12. J. L. Amey, J. Keeley, T. Choudhury, and I. Kuprov, Neural network interpretation using descrambler groups, Proc. Natl. Acad. Sci. 118, e2016917118 (2021).
  13. R. D. King, J. Rowland, S. G. Oliver, M. Young, W. Aubrey, E. Byrne, M. Liakata, M. Markham, P. Pir, L. N. Soldatova et al., The automation of science, Science 324, 85 (2009).
  14. J. Bongard and H. Lipson, Automated reverse engineering of nonlinear dynamical systems, Proc. Natl. Acad. Sci. 104, 9943 (2007).
  15. M. Schmidt and H. Lipson, Distilling free-form natural laws from experimental data, Science 324, 81 (2009).
  16. E. Kaiser, J. N. Kutz, and S. L. Brunton, Proceedings of the 57th IEEE Conference on Decision and Control, Miami Beach, 2018 (IEEE, Piscataway, 2018), pp. 6415–6421.
  17. T. Wu and M. Tegmark, Toward an artificial intelligence physicist for unsupervised learning, Phys. Rev. E 100, 033311 (2019).
  18. H. Li, X.-q. Shi, M. Huang, X. Chen, M. Xiao, C. Liu, H. Chaté, and H. Zhang, Data-driven quantitative modeling of bacterial active nematics, Proc. Natl. Acad. Sci. 116, 777 (2019).
  19. K. Champion, B. Lusch, J. N. Kutz, and S. L. Brunton, Data-driven discovery of coordinates and governing equations, Proc. Natl. Acad. Sci. 116, 22445 (2019).
  20. R. Iten, T. Metger, H. Wilming, L. del Rio, and R. Renner, Discovering Physical Concepts with Neural Networks, Phys. Rev. Lett. 124, 010508 (2020).
  21. A. Decelle, V. Martin-Mayor, and B. Seoane, Learning a local symmetry with neural networks, Phys. Rev. E 100, 050102(R) (2019).
  22. Y.-i. Mototake, Interpretable conservation law estimation by deriving the symmetries of dynamics from trained deep neural networks, Phys. Rev. E 103, 033303 (2021).
  23. S. J. Wetzel, R. G. Melko, J. Scott, M. Panju, and V. Ganesh, Discovering symmetry invariants and conserved quantities by interpreting siamese neural networks, Phys. Rev. Research 2, 033499 (2020).
  24. Z. Liu and M. Tegmark, Machine Learning Conservation Laws from Trajectories, Phys. Rev. Lett. 126, 180604 (2021).
  25. Y. Sun, Deep Learning Face Representation by Joint Identification-Verification (The Chinese University of Hong Kong, Hong Kong, 2015).
  26. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, in 37th International Conference on Machine Learning (ICML 2020), Vienna, Austria (PMLR, 2020), pp. 1597–1607.
  27. P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, Supervised contrastive learning, in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, virtual (2020).
  28. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.3.L042035 for the proof for the NV loss, model results with different constants and data conditions, application of symbolic regression, and details about dataset constructions.
  29. H. Goldstein, C. Poole, and J. Safko, Classical Mechanics (Addison Wesley, San Francisco, 2002).
  30. D. Misra, Mish: A self regularized non-monotonic neural activation function, arXiv:1908.08681.
  31. D. P. Kingma and J. Ba, in 3rd International Conference on Learning Representations, San Diego, 2015, edited by Y. Bengio and Y. LeCun (ICLR, La Jolla, 2015).
  32. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., in Advances in Neural Information Processing Systems 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Curran, Red Hook, 2019), pp. 8026–8037.
  33. Y. Takeuchi, Global Dynamical Properties of Lotka-Volterra Systems (World Scientific, Singapore, 1996).
  34. S. Ha and H. Jeong, nokpil/conservnet, http://doi.org/10.5281/zenodo.4491096 (CERN, Meyrin, 2021), version v1.0.0.
  35. Q. Xie, M.-T. Luong, E. Hovy, and Q. V. Le, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, 2020), pp. 10687–10698.
  36. R. G. Lopes, D. Yin, B. Poole, J. Gilmer, and E. D. Cubuk, Improving robustness without sacrificing accuracy with patch gaussian augmentation, arXiv:1906.02611 (2021).
  37. K. Xu, M. Zhang, J. Li, S. S. Du, K. Kawarabayashi and S. Jegelka, How neural networks extrapolate From feedforward to graph neural networks, in 9th International Conference on Learning Representations, Vienna, Austria, edited by Y. Bengio and Y. LeCun (ICLR, La Jolla, 2021).
  38. P. E. Kloeden and M. Rasmussen, Nonautonomous Dynamical Systems (American Mathematical Society, Providence, 2011), Vol. 176.
  39. F. T. Hioe and J. H. Eberly, N-Level Coherence Vector and Higher Conservation Laws in Quantum Optics and Quantum Mechanics, Phys. Rev. Lett. 47, 838 (1981).
  40. Y. Aharonov, S. Popescu, and D. Rohrlich, On conservation laws in quantum mechanics, Proc. Natl. Acad. Sci. 118, e1921529118 (2021).
  41. N. H. Christ, Conservation-law violation at high energy by anomalies, Phys. Rev. D 21, 1591 (1980).
  42. B. P. Abbott, R. Abbott, T. Abbott, M. Abernathy, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. Adhikari et al., Observation of Gravitational Waves from a Binary Black Hole Merger, Phys. Rev. Lett. 116, 061102 (2016).
  43. G. Aad, T. Abajyan, B. Abbott, J. Abdallah, S. A. Khalek, A. A. Abdelalim, R. Aben, B. Abi, M. Abolins, O. AbouZeid et al., Observation of a new particle in the search for the standard model Higgs boson with the ATLAS detector at the LHC, Phys. Lett. B 716, 1 (2012).

Outline

Information

Phys. Rev. Research 3, L042035– Published 9 December, 2021
  • Received 8 February 2021
  • Revised 7 June 2021
  • Accepted 4 November 2021
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation