- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Exploiting chaotic dynamics as deep neural networks

Shuhong Liu1,*, Nozomi Akashi2, Qingyao Huang1, Yasuo Kuniyoshi1, and Kohei Nakajima1,†

  • 1Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
  • 2Graduate School of Informatics, Kyoto University, Yoshida-honmachi, Sakyo-ku, Kyoto 606-8501, Japan

  • *Contact author: s-liu@isi.imi.i.u-tokyo.ac.jp
  • †Contact author: k-nakajima@isi.imi.i.u-tokyo.ac.jp
PDF

Phys. Rev. Research 7, 033031 – Published 8 July, 2025

DOI: https://doi.org/10.1103/l8dq-mphg

Abstract

Chaos presents complex dynamics arising from nonlinearity and a sensitivity to initial states. These characteristics suggest a depth of expressivity that underscores their potential for advanced computational applications. However, strategies to effectively exploit chaotic dynamics for information processing have largely remained elusive. In this study, we reveal that the essence of chaos can be found in various state-of-the-art deep neural networks. Drawing inspiration from this revelation, we propose a novel method that directly leverages chaotic dynamics for deep learning architectures. Our approach is systematically evaluated across distinct chaotic systems. In all instances, our framework presents superior results to conventional deep neural networks in terms of accuracy, convergence speed, and efficiency. Furthermore, we find an active role of transient chaos formation in our framework. Collectively, this study opens up a new path for the integration of chaos, which has long been overlooked in information processing, and provides insights into the potential fusion of chaotic dynamics within the domains of machine learning and neuromorphic computation.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (51)

  1. E. N. Lorenz, Deterministic nonperiodic flow, J. Atmos. Sci. 20, 130 (1963).
  2. Z. Yang, S. Zhang, and Y. C. Li, Chaotic dynamics of spin-valve oscillators, Phys. Rev. Lett. 99, 134101 (2007).
  3. S. J. Schiff, K. Jerger, D. H. Duong, T. Chang, M. L. Spano, and W. L. Ditto, Controlling chaos in the brain, Nature (London) 370, 615 (1994).
  4. H. Korn and P. Faure, Is there chaos in the brain? II. Experimental evidence and related models, C.R. Biol. 326, 787 (2003).
  5. W. S. Pritchard and D. W. Duke, Measuring chaos in the brain-a tutorial review of EEG dimension estimation, Brain Cognit. 27, 353 (1995).
  6. G. Deco, V. K. Jirsa, and A. R. McIntosh, Emerging concepts for the dynamical organization of resting-state activity in the brain, Nat. Rev. Neurosci. 12, 43 (2011).
  7. J. Backus, Can programming be liberated from the von Neumann style? A functional style and its algebra of programs, Commun. ACM 21, 613 (1978).
  8. D. Marković, A. Mizrahi, D. Querlioz, and J. Grollier, Physics for neuromorphic computing, Nat. Rev. Phys. 2, 499 (2020).
  9. H. Jaeger and H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304, 78 (2004).
  10. W. Maass, T. Natschläger, and H. Markram, Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Comput. 14, 2531 (2002).
  11. K. Nakajima and I. Fischer, Reservoir Computing: Theory, Physical Implementations, and Applications (Springer, Singapore, 2021).
  12. S. Ishihara and K. Kaneko, Magic number 7±2 in networks of threshold dynamics, Phys. Rev. Lett. 94, 058102 (2005).
  13. A. M. Saxe, J. L. McClelland, and S. Ganguli, Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, in Proceedings of the International Conference on Learning Representations (ICLR) (Banff, AB, Canada, 2014).
  14. Y. Mototake and T. Ikegami, The dynamics of deep neural networks, in Proceedings of the Twentieth International Symposium on Artificial Life and Robotics (AROB20) (Beppu, Oita, Japan, 2015), Vol. 20.
  15. S. S. Schoenholz, J. Gilmer, S. Ganguli, and J. Sohl-Dickstein, Deep information propagation, in Proceedings of the International Conference on Learning Representations (ICLR) (Toulon, France, 2017).
  16. B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein, and S. Ganguli, Exponential expressivity in deep neural networks through transient chaos, Advances in Neural Information Processing Systems 29, 3360 (2016).
  17. H. Lin, C. Wang, W. Yao, and Y. Tan, Chaotic dynamics in a neural network with different types of external stimuli, Commun. Nonlinear Sci. Numer. Simul. 90, 105390 (2020).
  18. C. Keup, T. Kühn, D. Dahmen, and M. Helias, Transient chaotic dimensionality expansion by recurrent networks, Phys. Rev. X 11, 021064 (2021).
  19. K. Inoue, S. Ohara, Y. Kuniyoshi, and K. Nakajima, Transient chaos in bidirectional encoder representations from transformers, Phys. Rev. Res. 4, 013204 (2022).
  20. R. Engelken, F. Wolf, and L. F. Abbott, Lyapunov spectra of chaotic recurrent neural networks, Phys. Rev. Res. 5, 043044 (2023).
  21. See Supplemental Material at http://link.aps.org/supplemental/10.1103/l8dq-mphg for additional literature reviews, analysis of transient dynamics in neural networks, implementation specifics, and supplementary experimental results.
  22. M. Kondo, S. Sunada, and T. Niiyama, Lyapunov exponent analysis for multilayer neural networks, Nonlinear Theory Appl. IEICE 12, 674 (2021).
  23. L. Storm, H. Linander, J. Bec, K. Gustavsson, and B. Mehlig, Finite-time Lyapunov exponents of deep neural networks, Phys. Rev. Lett. 132, 057301 (2024).
  24. K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, Las Vegas, NV, USA, 2016), pp. 770–778.
  25. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Miami, FL, USA, 2009), pp. 248–255.
  26. A. Krizhevsky and G. Hinton, Learning multiple layers of features from tiny images, Master's thesis, University of Toronto, Toronto, ON, Canada, 2009.
  27. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (Association for Computational Linguistics, Minneapolis, MN, USA, 2019), pp. 4171–4186.
  28. I. Turc, M.-W. Chang, K. Lee, and K. Toutanova, Well-read students learn better: On the importance of pre-training compact models, arXiv:1908.08962v2.
  29. A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, Learning word vectors for sentiment analysis, in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT) (Association for Computational Linguistics, Portland, OR, USA, 2011), pp. 142–150.
  30. M. Cernansky and M. Makula, Feed-forward echo state networks, in Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN) (IEEE, Montreal, QC, Canada, 2005), Vol. 3, pp. 1479–1482.
  31. B. Igelnik and Y.-H. Pao, Stochastic choice of basis functions in adaptive function approximation and the functional-link net, IEEE Trans. Neural Networks 6, 1320 (1995).
  32. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, Extreme learning machine: A new learning scheme of feedforward neural networks, in Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN) (IEEE, Budapest, Hungary, 2004), Vol. 2, pp. 985–990.
  33. E. Galaris, G. Fabiani, I. Gallos, I. Kevrekidis, and C. Siettos, Numerical bifurcation analysis of PDEs from lattice boltzmann model simulations: A parsimonious machine learning approach, J. Sci. Comput. 92, 34 (2022).
  34. G. Fabiani, E. Galaris, L. Russo, and C. Siettos, Parsimonious physics-informed random projection neural networks for initial value problems of ODEs and index-1 DAEs, Chaos 33, 043128 (2023).
  35. H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
  36. D. Verstraeten, B. Schrauwen, M. d'Haene, and D. Stroobandt, An experimental unification of reservoir computing methods, Neural Networks 20, 391 (2007).
  37. E. N. Lorenz, Predictability: A problem partly solved, in Predictability of Weather and Climate, edited by T. Palmer and R. Hagedorn (Cambridge University Press, Cambridge, UK, 2006), pp. 40–58.
  38. L. S. Pontryagin, Mathematical Theory of Optimal Processes, 1st ed. (CRC press, Routledge, London, UK, 1987).
  39. R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, Neural ordinary differential equations, in Advances in Neural Information Processing Systems, edited by S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Curran Associates, Inc., Montreal, QC, Canada, 2018), Vol. 31, pp. 6571–6583.
  40. J. Torrejon, M. Riou, F. A. Araujo, S. Tsunegi, G. Khalsa, D. Querlioz, P. Bortolotti, V. Cros, K. Yakushiji, A. Fukushima et al., Neuromorphic computing with nanoscale spintronic oscillators, Nature (London) 547, 428 (2017).
  41. S. Tsunegi, T. Taniguchi, K. Nakajima, S. Miwa, K. Yakushiji, A. Fukushima, S. Yuasa, and H. Kubota, Physical reservoir computing based on spin torque oscillator with forced synchronization, Appl. Phys. Lett. 114, 164101 (2019).
  42. J. Grollier, D. Querlioz, K. Camsari, K. Everschor-Sitte, S. Fukami, and M. D. Stiles, Neuromorphic spintronics, Nat. Electron. 3, 360 (2020).
  43. Y. Imai, S. Liu, N. Akashi, and K. Nakajima, Gradient-based optimization of spintronic devices, Appl. Phys. Lett. 126, 082402 (2025).
  44. T. Yamaguchi, N. Akashi, K. Nakajima, S. Tsunegi, H. Kubota, and T. Taniguchi, Synchronization and chaos in a spin-torque oscillator with a perpendicularly magnetized free layer, Phys. Rev. B 100, 224422 (2019).
  45. N. Akashi, T. Yamaguchi, S. Tsunegi, T. Taniguchi, M. Nishida, R. Sakurai, Y. Wakao, and K. Nakajima, Input-driven bifurcations and information processing capacity in spintronics reservoirs, Phys. Rev. Res. 2, 043303 (2020).
  46. T. Taniguchi, N. Akashi, H. Notsu, M. Kimura, H. Tsukahara, and K. Nakajima, Chaos in nanomagnet via feedback current, Phys. Rev. B 100, 174425 (2019).
  47. A. Kamimaki, T. Kubota, S. Tsunegi, K. Nakajima, T. Taniguchi, J. Grollier, V. Cros, K. Yakushiji, A. Fukushima, S. Yuasa et al., Chaos in spin-torque oscillator with feedback circuit, Phys. Rev. Res. 3, 043216 (2021).
  48. N. Akashi, Y. Kuniyoshi, S. Tsunegi, T. Taniguchi, M. Nishida, R. Sakurai, Y. Wakao, K. Kawashima, and K. Nakajima, A coupled spintronics neuromorphic approach for high-performance reservoir computing, Adv. Intell. Syst. 4, 2200123 (2022).
  49. N. Leroux, A. D. Riz, D. Sanz-Hernández, D. Marković, A. Mizrahi, and J. Grollier, Convolutional neural networks with radio-frequency spintronic nano-devices, Neuromorph. Comput. Eng. 2, 034002 (2022).
  50. B. U. Kesgin and U. Teğin, Photonic Neural Networks with Spatiotemporal Chaos in Multimode Fibers, Nanophotonics (Walter de Gruyterm, Berlin, 2025).
  51. https://github.com/ShuhongLL/Exploiting-Chaotic-Dynamics-as-DNNs.

Outline

Information

Phys. Rev. Research 7, 033031– Published 8 July, 2025
  • Received 25 November 2024
  • Accepted 3 June 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation