- Open Access
Connectivity Structure and Dynamics of Nonlinear Recurrent Neural Networks
Phys. Rev. X 15, 041019 – Published 3 November, 2025
DOI: https://doi.org/10.1103/2jt7-c8cq
Abstract
Studies of the dynamics of nonlinear recurrent neural networks often assume independent and identically distributed couplings, but large-scale connectomics data indicate that biological neural circuits exhibit markedly different connectivity properties. These include rapidly decaying singular-value spectra and structured singular-vector overlaps. Here, we develop a theory to analyze how these forms of structure shape high-dimensional collective activity in nonlinear recurrent neural networks. We first introduce the random-mode model, a random-matrix ensemble related to the singular-value decomposition that enables control over the spectrum and right-left mode overlaps. Then, using a novel path-integral calculation, we derive analytical expressions that reveal how connectivity structure affects features of collective dynamics: the dimension of activity, which quantifies the number of high-variance collective-activity fluctuations, and the temporal correlations that characterize the timescales of these fluctuations. We show that connectivity structure can be invisible in single-neuron activities, while dramatically shaping collective activity. Furthermore, despite the nonlinear, high-dimensional nature of these networks, the dimension of activity depends on just two connectivity parameters—the variance of the couplings and the effective rank of the coupling matrix, which quantifies the number of dominant rank-one connectivity components. We contrast the effects of single-neuron heterogeneity and low-dimensional connectivity, making predictions about how -scoring data affects the dimension of activity. Finally, we demonstrate the presence of structured overlaps between left and right modes in the Drosophila connectome, incorporate them into the theory, and show how they further shape collective dynamics.
Physics Subject Headings (PhySH)
Popular Summary
Understanding how patterns of connections in the brain shape neural activity is a major challenge in neuroscience. Brain networks can be mathematically broken down into input and output modes, and recent studies show that actual neural circuits are far from random, dominated by a small number of strong components. At the same time, new technologies allow us to record the activity of thousands of neurons at once, giving us a chance to directly link connectivity to collective neural dynamics. In this study, we show that the structure of connectivity can predict key features of network activity, including the number of independent activity patterns the network can generate (the activity dimension).
To achieve this, we develop a mathematical framework that models networks as collections of input and output modes with adjustable strengths. Using this approach, we derive how the connectivity structure determines the activity dimension and other collective properties. Analysis of our theory suggests that large-scale data can reveal relationships between network connectivity and the activity of thousands of neurons simultaneously.
Our results show that even when observing individual neurons, the underlying connectivity may appear hidden, but population-level recordings reveal its influence. In particular, just two measures—the typical connection strength and the number of dominant components—largely determine the activity dimension in uncorrelated modes. The framework also captures additional constraints when input-output modes are correlated, as seen in the fruit fly connectome.
This work provides a powerful tool to link detailed structural maps of neural circuits with their emergent activity patterns, offering a foundation for future studies on how connectivity shapes computation in the brain.
Article Text
References (105)
- O. Barak, D. Sussillo, R. Romo, M. Tsodyks, and L. F. Abbott, From fixed points to chaos: Three models of delayed discrimination, Prog. Neurobiol. 103, 214 (2013).
- V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, Context-dependent computation by recurrent dynamics in prefrontal cortex, Nature (London) 503, 78 (2013).
- D. Sussillo, M. M. Churchland, M. T. Kaufman, and K. V. Shenoy, A neural network that finds a naturalistic solution for the production of muscle activity, Nat. Neurosci. 18, 1025 (2015).
- S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9, 1735 (1997).
- I. Sutskever, O. Vinyals, and Q. V. Le, Sequence to sequence learning with neural networks, Adv. Neural Inf. Process. Syst. 27, 3104 (2014).
- K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder–decoder for statistical machine translation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), edited by A. Moschitti, B. Pang, and W. Daelemans (Association for Computational Linguistics, Doha, Qatar, 2014), pp. 1724–1734.
- K. Krishnamurthy, T. Can, and D. J. Schwab, Theory of gating in recurrent neural networks, Phys. Rev. X 12, 011011 (2022).
- K. Rajan, L. F. Abbott, and H. Sompolinsky, Stimulus-dependent suppression of chaos in recurrent neural networks, Phys. Rev. E 82, 011903 (2010).
- R. Engelken, F. Wolf, and L. F. Abbott, Lyapunov spectra of chaotic recurrent neural networks, Phys. Rev. Res. 5, 043044 (2023).
- H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
- D. Martí, N. Brunel, and S. Ostojic, Correlations between synapses in pairs of neurons slow down dynamics in randomly connected neural networks, Phys. Rev. E 97, 062314 (2018).
- D. Sussillo and L. F. Abbott, Generating coherent patterns of activity from chaotic neural networks, Neuron 63, 544 (2009).
- R. Laje and D. V. Buonomano, Robust timing and motor patterns by taming chaos in recurrent neural networks, Nat. Neurosci. 16, 925 (2013).
- J. Aljadeff, M. Stern, and T. Sharpee, Transition to chaos in random networks with cell-type-specific connectivity, Phys. Rev. Lett. 114, 088101 (2015).
- B. DePasquale, C. J. Cueva, K. Rajan, G. S. Escola, and L. F. Abbott, full-force: A target-based method for training recurrent networks, PLoS One 13, e0191527 (2018).
- T. Asabuki and C. Clopath, Taming the chaos gently: A predictive alignment learning rule in recurrent neural networks, Nat. Commun. 16, 6784 (2025).
- L. K. Scheffer, C. S. Xu, M. Januszewski, Z. Lu, S.-y. Takemura, K. J. Hayworth, G. B. Huang, K. Shinomiya, J. Maitlin-Shepard, S. Berg et al., A connectome and analysis of the adult drosophila central brain, eLife 9, e57443 (2020).
- S. Loomba, J. Straehle, V. Gangadharan, N. Heike, A. Khalifa, A. Motta, N. Ju, M. Sievers, J. Gempt, H. S. Meyer et al., Connectomic comparison of mouse and human cortex, Science 377, eabo0924 (2022).
- A. Shapson-Coe, M. Januszewski, D. R. Berger, A. Pope, Y. Wu, T. Blakely, R. L. Schalek, P. H. Li, S. Wang, J. Maitin-Shepard et al., A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution, Science 384, eadk4858 (2024).
- The MICrONS Consortium, Functional connectomics spanning multiple areas of mouse visual cortex, Nature (London) 640, 435 (2025).
- M. R. Tavakoli, J. Lyudchik, M. Januszewski, V. Vistunou, N. Agudelo Dueñas, J. Vorlaufer, C. Sommer, C. Kreuzinger, B. Oliveira, A. Cenameri et al., Light-microscopy-based connectomic reconstruction of mammalian brain tissue, Nature (London) 642, 398 (2025).
- S. Song, P. J. Sjöström, M. Reigl, S. Nelson, and D. B. Chklovskii, Highly nonrandom features of synaptic connectivity in local cortical circuits, PLOS Biol. 3, e68 (2005).
- V. Thibeault, A. Allard, and P. Desrosiers, The low-rank hypothesis of complex systems, Nat. Phys. 20, 294 (2024).
- I. Akjouj, M. Barbier, M. Clenet, W. Hachem, M. Maïda, F. Massol, J. Najim, and V. C. Tran, Complex systems in ecology: A guided tour with large Lotka–Volterra models and random matrices, Proc. R. Soc. A 480, 20230284 (2024).
- Z. Wang, W. Mai, Y. Chai, K. Qi, H. Ren, C. Shen, S. Zhang, G. Tan, Y. Hu, and Q. Wen, The geometry and dimensionality of brain-wide activity, eLife 14, RP100666 (2025).
- L. Pezon, V. Schmutz, and W. Gerstner, Linking neural manifolds to circuit structure in recurrent networks, bioRxiv 2024 (2024).
- F. Schuessler, F. Mastrogiuseppe, A. Dubreuil, S. Ostojic, and O. Barak, The interplay between randomness and structure during learning in RNNs, Adv. Neural Inf. Process. Syst. 33, 13352 (2020).
- C. H. Martin and M. W. Mahoney, Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning, J. Mach. Learn. Res. 22, 1 (2021).
- D. G. Clark, L. F. Abbott, and H. Sompolinsky, Symmetries and continuous attractors in disordered neural circuits, bioRxiv 2025 (2025).
- E. M. Trautmann, J. K. Hesse, G. M. Stine, R. Xia, S. Zhu, D. J. O’Shea, B. Karsh, J. Colonell, F. F. Lanfranchi, S. Vyas et al., Large-scale high-density brain-wide neural recording in nonhuman primates, Nat. Neurosci. 28, 1562 (2025).
- C. Stringer, M. Pachitariu, N. Steinmetz, M. Carandini, and K. D. Harris, High-dimensional geometry of population responses in visual cortex, Nature (London) 571, 361 (2019).
- C. Stringer, M. Pachitariu, N. Steinmetz, C. B. Reddy, M. Carandini, and K. D. Harris, Spontaneous behaviors drive multidimensional, brainwide activity, Science 364, eaav7893 (2019).
- J. Manley, S. Lu, K. Barber, J. Demas, H. Kim, D. Meyer, F. M. Traub, and A. Vaziri, Simultaneous, cortex-wide dynamics of up to 1 million neurons reveal unbounded scaling of dimensionality with neuron number, Neuron 112, 1694 (2024).
- S. Chung and L. F. Abbott, Neural population geometry: An approach for understanding biological and artificial neural networks, Curr. Opin. Neurobiol. 70, 137 (2021).
- J. P. Cunningham and M. Y. Byron, Dimensionality reduction for large-scale neural recordings, Nat. Neurosci. 17, 1500 (2014).
- P. Gao and S. Ganguli, On simplicity and complexity in the brave new world of large-scale neuroscience, Curr. Opin. Neurobiol. 32, 148 (2015).
- E. M. Trautmann, S. D. Stavisky, S. Lahiri, K. C. Ames, M. T. Kaufman, D. J. O’Shea, S. Vyas, X. Sun, S. I. Ryu, S. Ganguli et al., Accurate estimation of neural population dynamics without spike sorting, Neuron 103, 292 (2019).
- L. Meshulam and W. Bialek, Statistical mechanics for networks of real neurons, Rev. Mod. Phys. (to be published), 10.1103/jcrn-3nrc.
- D. G. Clark, L. F. Abbott, and A. Litwin-Kumar, Dimension of activity in random neural networks, Phys. Rev. Lett. 131, 118401 (2023).
This random structure, rather than exact, deterministic orthonormality, offers analytical tractability by eliminating the need to enforce orthonormality—a complicated global constraint; but see also Ref. [41], which handles random orthonormal matrices in a regression setting.
- A. Ingrosso, Optimal learning with excitatory and inhibitory synapses, PLoS Comput. Biol. 16, e1008536 (2020).
- F. Mastrogiuseppe and S. Ostojic, Linking connectivity, dynamics, and computations in low-rank recurrent neural networks, Neuron 99, 609 (2018).
- F. Schuessler, A. Dubreuil, F. Mastrogiuseppe, S. Ostojic, and O. Barak, Dynamics of random recurrent networks with correlated low-rank structure, Phys. Rev. Res. 2, 013111 (2020).
- M. Beiran, A. Dubreuil, A. Valente, F. Mastrogiuseppe, and S. Ostojic, Shaping dynamics with multiple populations in low-rank recurrent networks, Neural Comput. 33, 1572 (2021).
- A. Dubreuil, A. Valente, M. Beiran, F. Mastrogiuseppe, and S. Ostojic, The role of population structure in computations through neural dynamics, Nat. Neurosci. 25, 783 (2022).
The Gaussian assumption for individual components is not crucial—our results depend on only first and second moments for large due to the central limit theorem; higher-order cumulants (scaling appropriately with ) are subdominant in the path-integral saddle-point approximation.
- R. Vershynin, High-Dimensional Probability: An Introduction with Applications in Data Science (Cambridge University Press, Cambridge, England, 2018), Vol. 47.
This follows from using Wick’s theorem to evaluate the expectations of and for the numerator and denominator, respectively, of .
- J. Pennington, S. Schoenholz, and S. Ganguli, Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice, in Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA (2017), https://papers.nips.cc/paper_files/paper/2017/hash/d9fc0cdb67638d50f411432d0d41d0ba-Abstract.html.
- J. Pennington, S. Schoenholz, and S. Ganguli, The emergence of spectral universality in deep networks, in International Conference on Artificial Intelligence and Statistics (PMLR, 2018), pp. 1924–1932.
This behaves similarly to the more conventional but allows for analytical evaluation of Gaussian integrals.
This duality is reminiscent of ergodicity theorems, which relate temporal and spatial averages, but differs in a variety of ways, including that it holds even for finite-time intervals, whereas ergodicity concerns infinite-time limits.
- K. Rajan, L. F. Abbott, and H. Sompolinsky, Inferring stimulus selectivity from the spatial structure of neural network dynamics, Adv. Neural Inf. Process. Syst. 23 (2010).
- L. F. Abbott, K. Rajan, and H. Sompolinsky, Interactions between intrinsic and stimulus-evoked activity in recurrent neural networks, in The Dynamic Brain: An Exploration of Neuronal Variability and its Functional Significance, edited by M. Ding and D. Glanzman (Oxford University Press, New York, 2011), pp. 65–82.
- P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli, A theory of multineuronal dimensionality, dynamics and measurement, bioRxiv 214262 (2017).
- A. Litwin-Kumar, K. D. Harris, R. Axel, H. Sompolinsky, and L. F. Abbott, Optimal degrees of synaptic connectivity, Neuron 93, 1153 (2017).
- S. Recanatesi, G. K. Ocker, M. A. Buice, and E. Shea-Brown, Dimensionality in recurrent spiking networks: Global trends in activity and local origins in connectivity, PLoS Comput. Biol. 15, 1 (2019).
- S. Recanatesi, S. Bradde, V. Balasubramanian, N. A. Steinmetz, and E. Shea-Brown, A scale-dependent measure of system dimensionality, Patterns 3, 100555 (2022).
- Y. Hu and H. Sompolinsky, The spectrum of covariance matrices of randomly connected recurrent neuronal networks with linear dynamics, PLoS Comput. Biol. 18, 1 (2022).
- A. Crisanti and H. Sompolinsky, Path integral approach to random neural networks, Phys. Rev. E 98, 062120 (2018).
- M. Helias and D. Dahmen, Statistical Field Theory for Neural Networks, Lecture Notes in Physics (Springer, Cham, 2020), 10.1007/978-3-030-46444-8.
- W. Zou and H. Huang, Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings, SciPost Phys. Lect. Notes 079 (2024).
- P. C. Martin, E. D. Siggia, and H. A. Rose, Statistical dynamics of classical systems, Phys. Rev. A 8, 423 (1973).
- C. De Dominicis, Dynamics as a substitute for replicas in systems with quenched random impurities, Phys. Rev. B 18, 4913 (1978).
- J. A. Hertz, Y. Roudi, and P. Sollich, Path integral methods for the dynamics of stochastic and disordered systems, J. Phys. A 50, 033001 (2016).
- T. Hromádka, M. R. DeWeese, and A. M. Zador, Sparse representation of sounds in the unanesthetized auditory cortex, PLOS Biol. 6, e16 (2008).
This symmetry arises because the dynamics of the network are determined by an effective coupling matrix , where and . Swapping the distributions of and does not change the spectrum when and are i.i.d., since and have the same nonzero eigenvalues.
- B. Rider and C. D. Sinclair, Extremal laws for the real ginibre ensemble, Ann. Appl. Probab. 24, 1621 (2014).
- D. G. Clark, Transient dynamics of associative memory models, arXiv:2506.05303.
If model neurons represent clusters of biological neurons, strong self-connections may instead be interpreted as within-cluster coupling [71, 72].
- M. Stern, H. Sompolinsky, and L. F. Abbott, Dynamics of random neural networks with bistable units, Phys. Rev. E 90, 062710 (2014).
- A. Litwin-Kumar and B. Doiron, Slow dynamics and high variability in balanced cortical networks with clustered connections, Nat. Neurosci. 15, 1498 (2012).
- R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon, Network motifs: simple building blocks of complex networks, Science 298, 824 (2002).
- Y. Hu, J. Trousdale, K. Josić, and E. Shea-Brown, Motif statistics and spike correlations in neuronal networks, J. Stat. Mech. (2013) P03012.
- P. V. Aceituno, T. Rogers, and H. Schomerus, Universal hypotrochoidic law for random matrices with cyclic correlations, Phys. Rev. E 100, 010302(R) (2019).
- E. S. Schaffer, N. Mishra, M. R. Whiteway, W. Li, M. B. Vancura, J. Freedman, K. B. Patel, V. Voleti, L. Paninski, E. M. Hillman et al., The spatial and temporal structure of neural activity across the fly brain, Nat. Commun. 14, 5572 (2023).
- E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, Weak pairwise correlations imply strongly correlated network states in a neural population, Nature (London) 440, 1007 (2006).
- O. Maoz, G. Tkačik, M. S. Esteki, R. Kiani, and E. Schneidman, Learning probabilistic neural representations with randomly connected circuits, Proc. Natl. Acad. Sci. U.S.A. 117, 25066 (2020).
- E. Gardner, The space of interactions in neural network models, J. Phys. A 21, 257 (1988).
- B. Bordelon, J. Zavatone-Veth, and C. Pehlevan, Mean field theory of representation learning in large rnns, in Cosyne Abstracts 2024 (2024) abstract 3-038.
- G. R. Yang, M. R. Joglekar, H. F. Song, W. T. Newsome, and X.-J. Wang, Task representations in neural networks trained to perform many cognitive tasks, Nat. Neurosci. 22, 297 (2019).
- L. N. Driscoll, K. Shenoy, and D. Sussillo, Flexible multitask computation in recurrent networks utilizes shared dynamical motifs, Nat. Neurosci. 27, 1349 (2024).
- E. Marder and J.-M. Goaillard, Variability, compensation and homeostasis in neuron and network function, Nat. Rev. Neurosci. 7, 563 (2006).
- C. Teeter, R. Iyer, V. Menon, N. Gouwens, D. Feng, J. Berg, A. Szafer, N. Cain, H. Zeng, M. Hawrylycz et al., Generalized leaky integrate-and-fire models classify multiple neuron types, Nat. Commun. 9, 709 (2018).
- B. K. Murphy and K. D. Miller, Balanced amplification: A new mechanism of selective amplification of neural activity patterns, Neuron 61, 635 (2009).
- G. Hennequin, T. P. Vogels, and W. Gerstner, Optimal control of transient dynamics in balanced networks supports generation of complex movements, Neuron 82, 1394 (2014).
- K. Rajan and L. F. Abbott, Eigenvalue spectra of random matrices for neural networks, Phys. Rev. Lett. 97, 188104 (2006).
- J. Kadmon and H. Sompolinsky, Transition to chaos in random neuronal networks, Phys. Rev. X 5, 041030 (2015).
- L. Tiberi, D. Dahmen, and M. Helias, Hidden connectivity structures control collective network dynamics, arXiv:2303.02476.
- V. Schmutz, J. Brea, and W. Gerstner, Emergent rate-based dynamics in duplicate-free populations of spiking neurons, Phys. Rev. Lett. 134, 018401 (2025).
- Z. Lin and H. Huang, Spiking mode-based neural networks, Phys. Rev. E 110, 024306 (2024).
- N. Eckstein, A. S. Bates, A. Champion, M. Du, Y. Yin, P. Schlegel, A. K.-Y. Lu, T. Rymer, S. Finley-May, T. Paterson et al., Neurotransmitter classification from electron microscopy images at synaptic sites in drosophila melanogaster, Cell 187, 2574 (2024).
- W. W. Liu and R. I. Wilson, Glutamate is an inhibitory neurotransmitter in the drosophila olfactory system, Proc. Natl. Acad. Sci. U.S.A. 110, 10294 (2013).
- S. Molina-Obando, J. F. Vargas-Fique, M. Henning, B. Gür, T. M. Schladt, J. Akhtar, T. K. Berger, and M. Silies, On selectivity in the drosophila visual system is a multisynaptic process involving both glutamatergic and gabaergic inhibition, eLife 8, e49373 (2019).
- O. Roy and M. Vetterli, The effective rank: A measure of effective dimensionality, in 2007 15th European Signal Processing Conference, Poznan, Poland (IEEE, 2007), pp. 606–610.
- G. Christodoulou, T. P. Vogels, and E. J. Agnes, Regimes and mechanisms of transient amplification in abstract and biological neural networks, PLoS Comput. Biol. 18, e1010365 (2022).
- M. Mézard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications (World Scientific Publishing Company, Singapore, 1987), Vol. 9.
- R. Price, A useful theorem for nonlinear devices having Gaussian inputs, IRE Trans. Inf. Theory 4, 69 (1958).
- A. van Meegen, T. Kühn, and M. Helias, Large-deviation approach to random recurrent neuronal networks: Parameter inference and fluctuation-induced transitions, Phys. Rev. Lett. 127, 158302 (2021).
- M. A. Buice and J. D. Cowan, Field-theoretic approach to fluctuation effects in neural networks, Phys. Rev. E 75, 051919 (2007).
Note that the variance of is scaled by ; equivalently, we could factor out from the variance and include as a multiplicative factor in the latent-variable single-site dynamics [Eq. (i17)].
- J. P. Stroud, M. A. Porter, G. Hennequin, and T. P. Vogels, Motor primitives in space and time via targeted gain modulation in cortical networks, Nat. Neurosci. 21, 1774 (2018).
- M. Mézard, The space of interactions in neural networks: Gardner’s computation with the cavity method, J. Phys. A 22, 2181 (1989).
- M. Mézard, Mean-field message-passing equations in the Hopfield model and its generalizations, Phys. Rev. E 95, 022117 (2017).
- F. Mastrogiuseppe, From dynamics to computations in recurrent neural networks, Ph.D. thesis, Université Paris sciences et lettres, 2017.
