- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Dynamical learning in deep asymmetric recurrent neural networks

Davide Badalotti, Carlo Baldassi, Marc Mézard, Mattia Scardecchia, and Riccardo Zecchina*

  • *Contact author: riccardo.zecchina@unibocconi.it
PDF

Phys. Rev. Research 8, 023281 – Published 11 June, 2026

DOI: https://doi.org/10.1103/lbr6-91sn

Abstract

We investigate recurrent neural networks with asymmetric interactions and demonstrate that the inclusion of self-couplings or sparse excitatory intermodule connections leads to the emergence of a densely connected manifold of dynamically accessible stable configurations. This representation manifold is exponentially large in system size and is reachable through simple local dynamics, despite constituting a subdominant subset of the global configuration space. We further show that learning can be implemented directly on this structure via a fully local, gradient-free mechanism that selectively stabilizes a single task-relevant network configuration. Unlike error-driven or contrastive learning schemes, this approach does not require explicit comparisons between network states obtained with and without output supervision. Instead, transient supervisory signals bias the dynamics toward the representation manifold, after which local plasticity consolidates the attained configuration, effectively shaping the latent representation space. Numerical evaluations on standard image classification benchmarks indicate performance comparable to that of multilayer perceptrons trained using backpropagation. More generally, these results suggest that the dynamical accessibility of fixed points and the stabilization of internal network dynamics constitute viable alternative principles for learning in recurrent systems, with conceptual links to statistical physics and potential implications for biologically motivated and neuromorphic computing architectures.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (38)

  1. Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature (London) 521, 436 (2015).
  2. The new NeuroAI Panel, The new NeuroAI, Nat. Mach. Intell. 6, 245 (2024).
  3. A. Ororbia, A. Mali, A. Kohan, B. Millidge, and T. Salvatori, A review of neuroscience-inspired machine learning, arXiv:2403.18929.
  4. J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, in International Conference on Machine Learning (PMLR, Brookline, MA, 2015), pp. 2256–2265.
  5. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, 2017), Vol. 30, pp. 5998–6008.
  6. D.-H. Lee, S. Zhang, A. Fischer, and Y. Bengio, Difference target propagation, in Joint European Conference on Machine Learning and Knowledge Discovery in Databases (Springer, Cham, 2015), pp. 498–515.
  7. T. P. Lillicrap, D. Cownden, D. B. Tweed, and C. J. Akerman, Random synaptic feedback weights support error backpropagation for deep learning, Nat. Commun. 7, 13276 (2016).
  8. R. P. Rao and D. H. Ballard, Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects, Nat. Neurosci. 2, 79 (1999).
  9. B. Millidge, A. Seth, and C. L. Buckley, Predictive coding: A theoretical and experimental review, arXiv:2107.12979.
  10. G. Hinton, The forward-forward algorithm: Some preliminary investigations, arXiv:2212.13345.
  11. K. Nakajima and I. Fischer, Reservoir Computing (Springer, Cham, 2021).
  12. J. H. Lee, T. Delbruck, and M. Pfeiffer, Training deep spiking neural networks using backpropagation, Front. Neurosci. 10, 508 (2016).
  13. Z. Yu, W. Huang, and H. Huang, Neural Langevin machine: A local asymmetric learning rule can be creative, arXiv:2506.23546.
  14. B. Scellier and Y. Bengio, Equilibrium propagation: Bridging the gap between energy-based models and backpropagation, Front. Comput. Neurosci. 11, 24 (2017).
  15. H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
  16. M. Stern, H. Sompolinsky, and L. F. Abbott, Dynamics of random neural networks with bistable units, Phys. Rev. E 90, 062710 (2014).
  17. C. Baldassi, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, Subdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapses, Phys. Rev. Lett. 115, 128101 (2015).
  18. C. Baldassi, C. Borgs, J. T. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes, Proc. Natl. Acad. Sci. USA 113, E7655 (2016).
  19. C. Baldassi, C. Lauditi, E. M. Malatesta, G. Perugini, and R. Zecchina, Unveiling the structure of wide flat minima in neural networks, Phys. Rev. Lett. 127, 278301 (2021).
  20. D. Barbier, Finding the right path: Statistical mechanics of connected solutions in constraint satisfaction problems, arXiv:2505.20954.
  21. See Supplemental Material at http://link.aps.org/supplemental/10.1103/lbr6-91sn for detailed analytical derivations, extended learning algorithms, and additional experimental results, which includes Refs. [35, 36, 37, 38].
  22. D. Gamarnik, The overlap gap property: A topological barrier to optimizing over random structures, Proc. Natl. Acad. Sci. USA 118, e2108492118 (2021).
  23. D. Badalotti, C. Baldassi, M. Mezard, M. Scardecchia, and R. Zecchina, Dynamical learning schemes for deep learning, in preparation (unpublished).
  24. C. Baldassi, F. Gerace, H. J. Kappen, C. Lucibello, L. Saglietti, E. Tartaglione, and R. Zecchina, Role of synaptic stochasticity in training low-precision neural networks, Phys. Rev. Lett. 120, 268103 (2018).
  25. B. L. Annesi, C. Lauditi, C. Lucibello, E. M. Malatesta, G. Perugini, F. Pittorino, and L. Saglietti, Star-shaped space of solutions of the spherical negative perceptron, Phys. Rev. Lett. 131, 227301 (2023).
  26. L. Saglietti, F. Gerace, A. Ingrosso, C. Baldassi, and R. Zecchina, From statistical inference to a differential learning rule for stochastic neural networks, Interface Focus 8, 20180033 (2018).
  27. A. Alemi, C. Baldassi, N. Brunel, and R. Zecchina, A three-threshold learning rule approaches the maximal capacity of recurrent neural networks, PLoS Comput. Biol. 11, e1004439 (2015).
  28. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86, 2278 (1998).
  29. H. Xiao, K. Rasul, and R. Vollgraf, Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms, arXiv:1708.07747.
  30. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, ImageNet: A large-scale hierarchical image database, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Piscataway, NJ, 2009), pp. 248–255.
  31. D. Badalotti, M. Scardecchia, C. Baldassi, R. Zecchina, and M. Mezard, Willinki/darnax: v0.6.1, Zenodo (2026), https://doi.org/10.5281/zenodo.19496650.
  32. Y. Bengio, N. Léonard, and A. Courville, Estimating or propagating gradients through stochastic neurons for conditional computation, arXiv:1308.3432.
  33. H. Jaeger, The “echo state” approach to analysing and training recurrent neural networks—with an erratum note, Technical Report 148, German National Research Center for Information Technology, 2001.
  34. C. Baldassi, F. Pittorino, and R. Zecchina, Shaping the learning landscape in neural networks around wide flat minima, Proc. Natl. Acad. Sci. USA 117, 161 (2020).
  35. K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2016), pp. 770–778.
  36. E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, RandAugment: Practical data augmentation with no separate search, in Advances in Neural Information Processing Systems (2020), Vol. 33, pp. 16754–16765.
  37. S. Yun, D. Han, S. J. Oh, S. Chun, J. Choe, and Y. Yoo, CutMix: Regularization strategy to train strong classifiers with localizable features, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2019), pp. 6023–6032.
  38. H. Zhang, M. Cissé, Y. N. Dauphin, and D. Lopez-Paz, mixup: Beyond Empirical Risk Minimization, in International Conference on Learning Representations (ICLR, 2018).

Outline

Information

Phys. Rev. Research 8, 023281– Published 11 June, 2026
  • Received 9 September 2025
  • Revised 29 December 2025
  • Accepted 27 February 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation