- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Effective theory of collective deep learning

Lluís Arola-Fernández* and Lucas Lacasa†

  • *Contact author: lluisarolaf@gmail.com
  • †Contact author: lucas@ifisc.uib-csic.es
PDF

Phys. Rev. Research 6, L042040 – Published 12 November, 2024

DOI: https://doi.org/10.1103/PhysRevResearch.6.L042040

Abstract

Unraveling the emergence of collective learning in systems of coupled artificial neural networks points to broader implications for physics, machine learning, neuroscience, and society. Here we introduce a minimal model of interacting deep neural nets that condenses several recent decentralized algorithms by considering a competition between two terms: the local learning dynamics in the parameters of each neural network unit, and a diffusive coupling among units that tends to homogenize the parameters of the ensemble. We derive an effective theory for linear networks to show that the coarse-grained behavior of our system is equivalent to a deformed Ginzburg-Landau model with quenched disorder. This framework predicts depth-dependent disorder-order-disorder phase transitions in the parameters' solutions that reveal a depth-delayed onset of a collective learning phase and a low-rank microscopic learning path. We validate the theory in coupled ensembles of realistic neural networks trained on MNIST and CIFAR-10 datasets under privacy constraints. Interestingly, experiments confirm that individual networks–trained on private data–can fully generalize to unseen data classes when the collective learning phase emerges. Our work elucidates the physics of collective learning and contributes to the mechanistic interpretability of deep learning in decentralized settings.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (61)

  1. P. W. Anderson, More is different, Science 177, 393 (1972).
  2. M. Mézard, G. Parisi, and M. A. Virasoro, Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications, Vol. 9 (World Scientific Publishing Company, 1987).
  3. S. H. Strogatz, Sync: The Emerging Science of Spontaneous Order (Hyperion, New York, 2003).
  4. M. A. Muñoz, Colloquium: Criticality and dynamical scaling in living systems, Rev. Mod. Phys. 90, 031001 (2018).
  5. R. Solé, M. Moses, and S. Forrest, Liquid brains, solid brains, Phil. Trans. R. Soc. B 374, 20190040 (2019).
  6. J. Piñero and R. Solé, Statistical physics of liquid brains, Phil. Trans. R. Soc. B 374, 20180376 (2019).
  7. D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory (John Wiley, New York, 1949).
  8. M. J. Mataric, Designing emergent behaviors: From local interactions to collective intelligence, in PSICFAA2 (MIT Press, Cambridge, MA, 1993), pp. 432–441.
  9. D. Ha and Y. Tang, Collective intelligence for deep learning: A survey of recent developments, Collective Intelligence 1, 26339137221114874 (2022).
  10. A. Olsson, E. Knapska, and B. Lindström, The neural and computational systems of social learning, Nat. Rev. Neurosci. 21, 197 (2020).
  11. J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Natl. Acad. Sci. USA 79, 2554 (1982).
  12. J. Kennedy and R. Eberhart, Particle swarm optimization, in Proceedings of ICNN'95, Vol. 4 (IEEE, Perth, 1995), p. 1942.
  13. G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld, N. Tishby, L. Vogt-Maranto, and L. Zdeborova, Machine learning and the physical sciences, Rev. Mod. Phys. 91, 045002 (2019).
  14. M. Mézard, Spin glass theory and its new challenge: Structured disorder, Indian J. Phys. 98, 3757 (2024).
  15. Z. Liu and M. Ueda, Towards understanding grokking: An effective theory of representation learning, in Advances in Neural Information Processing System, Vol. 35 (NeurIPS, 2022), p. 34651.
  16. L. Ziyin and M. Ueda, Exact phase transitions in deep learning, arXiv:2205.12510.
  17. P. Mehta, M. Bukov, C.-H. Wang, A. G. R. Day, C. Richardson, C. K. Fisher, and D. J. Schwab, A high-bias, low-variance introduction to machine learning for physicists, Phys. Rep. 810, 1 (2019).
  18. D. A. Roberts, S. Yaida, and B. Hanin, The Principles of Deep Learning Theory (Cambridge University Press, Cambridge, MA, 2022).
  19. I. J. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT Press, Cambridge, MA, 2016).
  20. J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, A. Bridgland, C. Meyer, S. A. A. Kohl, A. J. Ballard, A. Cowie, B. Romera-Paredes, S. Nikolov, R. Jain, J. Adler, T. Back et al., Highly accurate protein structure prediction for the human proteome, Nature (London) 596, 583 (2021).
  21. N. Brown and T. Sandholm, Superhuman AI for multiplayer poker, Science 365, 885 (2019).
  22. S. Bubeck et al., Sparks of artificial general intelligence: Early experiments with GPT-4, arXiv:2303.12712.
  23. T. Ben-Nun and T. Hoefler, Demystifying parallel and distributed deep learning: An in-depth concurrency analysis, ACM Comput. Surv. 52, 1 (2019).
  24. T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, Federated learning: Challenges, methods, and future directions, IEEE Sig. P. Mag. 37, 50 (2019).
  25. S. Lee, S. Purushwalkam, M. Cogswell, D. Crandall, and D. Batra, Why M heads are better than one: Training a diverse ensemble of deep networks, arXiv:1511.06314.
  26. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas, Communication-efficient learning of deep networks from decentralized data, in Artificial intelligence and statistics (PMLR, 2017), pp. 1273–1282.
  27. P. P. Liang, T. Liu, L. Ziyin, N. B. Allen, R. P. Auerbach, D. Brent, R. Salakhutdinov, and L.-P. Morency, Think locally, act globally: Federated learning with local and global representations, arXiv:2001.01523.
  28. J. Wang and G. Joshi, Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms, J. Mach. Learn. Res. 22, 9709 (2021).
  29. F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, and H. Zhu, A comprehensive survey on transfer learning, Proc. IEEE 109, 43 (2020).
  30. S. Zhang, A. E. Choromanska, and Y. LeCun, Deep learning with elastic averaging SGD, Adv. neural inf. process. syst. 1, 685 (2015).
  31. B. E. Woodworth, K. K. Patel, and N. Srebro, Minibatch vs local SGD for heterogeneous distributed learning, Adv. Neural Inf. Process. Syst. 33, 6281 (2020).
  32. X. Li et al., A convergence theory for federated average: Beyond smoothness, in IEEE International Conference on Big Data (IEEE, Piscataway, NJ, 2022), pp. 1292–1297.
  33. B. Huang, X. Li, Z. Song, and X. Yang, FL-NKT: A neural tangent kernel-based framework for federated learning convergence analysis, arXiv:2105.05001.
  34. X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, On the convergence of FedAvg on non-IID data, arXiv:1907.02189.
  35. C. Baldassi, C. Borgs, J. T. Chayes, A. Ingrosso, C. Lucibello, L. Saglietti, and R. Zecchina, Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes, Proc. Natl. Acad. Sci. USA 113, E7655 (2016).
  36. C. Baldassi, F. Pittorino, and R. Zecchina, Shaping the learning landscape in neural networks around wide flat minima, Proc. Natl. Acad. Sci. USA 117, 161 (2020).
  37. A. M. Saxe, J. L. McClelland, and S. Ganguli, Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, arXiv:1312.6120.
  38. C. Van den Broeck, J. M. R. Parrondo, J. Armero, and A. Hernandez-Machado, Mean field model for spatially extended systems in the presence of multiplicative noise, Phys. Rev. E 49, 2639 (1994).
  39. J. Buceta, J. M. R. Parrondo, and F. J. dela Rubia, Random Ginzburg-Landau model revisited: Reentrant phase transitions, Phys. Rev. E 63, 031103 (2001).
  40. R. Toral, C. J. Tessone, and J. M. V. P. Lopes, Collective effects induced by diversity in extended systems, Eur. Phys. J.: Spec. Top. 143, 59 (2007).
  41. N. Komin, L. Lacasa, and R. Toral, Critical behavior of a Ginzburg–Landau model with additive quenched noise, J. Stat. Mech. (2010) P12008.
  42. P. Hohenberg and A. Krekhov, An introduction to the Ginzburg-Landau theory of phase transitions and nonequilibrium patterns, Phys. Rep. 572, 1 (2015).
  43. Too much coupling destroys the collective learning phase either by collective regularization or by an explosion of the diffusion term of Eq. (2) leading to divergences.
  44. See Supplemental Material at http://link.aps.org/supplemental/10.1103/PhysRevResearch.6.L042040 for the results on additional experiments with varying activation functions (Sec. A), the CIFAR-10 dataset and convolutional nets (Sec. B), and comparing the performance of the collective system against a single unit trained with global data (Sec. C).
  45. A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in Advances in Neural Information Processing Systems, Vol. 31 (NeurIPS, 2018).
  46. C. H. Martin and M. W. Mahoney, Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning, J. Mach. Learn. Res. 22, 1 (2021).
  47. A. Achille, M. Rovere, and S. Soatto, Critical learning periods in deep neural networks, arXiv:1711.08856.
  48. A. Sclocchi and M. Wyart, On the different regimes of stochastic gradient descent, Proc. Natl. Acad. Sci. USA 121, e2316301121 (2024).
  49. E. Ott and T. Antonsen, Low dimensional behavior of large systems of globally coupled oscillators, Chaos 18, 037113 (2008).
  50. C. Bick, M. Goodfellow, C. R. Laing, and E. A. Martens, Understanding the dynamics of biological and neural oscillator networks through exact mean-field reductions: A review, J. Math. Neurosc. 10, 9 (2020).
  51. G. Gottwald, Model reduction for networks of coupled oscillators, Chaos 25, 053111 (2015).
  52. L. Arola-Fernández, P. S. Skardal, and A. Arenas, Geometric unfolding of synchronization dynamics on networks, Chaos 31, 061105 (2021).
  53. M. Newman, Networks: An Introduction (Oxford University Press, Inc., New York, NY, 2010).
  54. J. Gómez-Gardeñes, S. Gomez, A. Arenas, and Y. Moreno, Explosive synchronization transitions in scale-free networks, Phys. Rev. Lett. 106, 128701 (2011).
  55. P. S. Skardal, J. G. Restrepo, and E. Ott, Frequency assortativity can induce chaos in oscillator networks, Phys. Rev. E 91, 060902(R) (2015).
  56. L. Arola-Fernández, S. Faci-Lázaro, P. S. Skardal, E.-C. Boghiu, J. Gómez-Gardeñes, and A. Arenas, Emergence of explosive synchronization bombs in networks of oscillators, Commun. Phys. 5, 264 (2022).
  57. K. Chatterjee, D. Zufferey, and M. A. Nowak, Evolutionary game dynamics in populations with different learners, J. Theor. Biol. 301, 161 (2012).
  58. D. Gao, X. Yao, and Q. Yang, A survey on heterogeneous federated learning, arXiv:2210.04505.
  59. J. Gómez-Gardeñes, D. Soriano-Paños, and A. Arenas, Critical regimes driven by recurrent mobility patterns of reaction–diffusion processes in networks, Nat. Phys. 14, 391 (2018).
  60. R. Vinuesa, H. Azizpour, I. Leite, M. Balaam, V. Dignum, S. Domisch, A. Felländer, S. D. Langhans, M. Tegmark, and F. F. Nerini, The role of artificial intelligence in achieving the sustainable development goals, Nat. Commun. 11, 233 (2020).
  61. github.com/mystic-blue/collective-learning.

Outline

Information

Phys. Rev. Research 6, L042040– Published 12 November, 2024
  • Received 9 November 2023
  • Accepted 16 October 2024
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation