- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

High-Dimensional Dynamics in Low-Dimensional Networks

Yue Wan and Robert Rosenbaum*

  • Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, Indiana 46556, USA

  • *Contact author: Robert.Rosenbaum@nd.edu
PDF

Phys. Rev. X 16, 031025 – Published 3 August, 2026

DOI: https://doi.org/10.1103/79h2-36r3

Abstract

Many networks in nature and applications have an approximate low-rank structure in the sense that their connectivity structure is dominated by a few dimensions. It is natural to expect that dynamics on such networks would also be low-dimensional. Indeed, theoretical results show that low-rank networks produce low-dimensional dynamics whenever the network is isolated from external perturbations or input. However, networks in nature are rarely isolated. Here, we study the dimensionality of dynamics in recurrent networks with low-dimensional structure driven by high-dimensional inputs or perturbations. We find that dynamics in such networks can be high- or low-dimensional, and we derive mathematical conditions on the network structure under which linearized dynamics are high-dimensional. In many low-rank networks, dynamics are suppressed in directions aligned with the network’s low-rank structure, a phenomenon we term “low-rank suppression.” We show that several low-rank network structures arising in nature satisfy the conditions for generating high-dimensional dynamics and low-rank suppression. Our results clarify important but counterintuitive relationships between a recurrent network’s connectivity structure and the structure of its response to external input.

View figure in article

Physics Subject Headings (PhySH)

Popular Summary

Many networks exhibit low-dimensional structure, meaning they can be described by fewer variables than the number of variables comprising the network. Likewise, their dynamics or activity can also be low dimensional. This study investigates how a network’s structural dimensionality relates to the dimensionality of its dynamics, and specifically, its response to external inputs or noise. Using mathematical analysis and computer simulations, we demonstrate that networks with low-dimensional structure can yield either high- or low-dimensional responses. This outcome depends heavily on the finer structure of the network and its input. Furthermore, we reveal that networks with low-dimensional structure are less sensitive to inputs or perturbations aligned to their low-dimensional structure and are more sensitive to random and misaligned inputs. Ultimately, these findings offer important implications for how we interpret low-dimensional structures in real-world networks.

Article Text

References (62)

  1. V. Thibeault, A. Allard, and P. Desrosiers, The low-rank hypothesis of complex systems, Nat. Phys. 20, 294 (2024).
  2. J. Aljadeff, D. Renfrew, M. Vegué, and T. O. Sharpee, Low-dimensional dynamics of structured random networks, Phys. Rev. E 93, 022302 (2016).
  3. F. Mastrogiuseppe and S. Ostojic, Linking connectivity, dynamics, and computations in low-rank recurrent neural networks, Neuron 99, 609 (2018).
  4. I. D. Landau and H. Sompolinsky, Macroscopic fluctuations emerge in balanced networks with incomplete recurrent alignment, Phys. Rev. Res. 3, 023171 (2021).
  5. A. Dubreuil, A. Valente, M. Beiran, F. Mastrogiuseppe, and S. Ostojic, The role of population structure in computations through neural dynamics, Nat. Neurosci. 25, 783 (2022).
  6. I. D. Landau and H. Sompolinsky, Coherent chaos in a recurrent neural network with structured connectivity, PLoS Comput. Biol. 14, e1006309 (2018).
  7. F. Schuessler, A. Dubreuil, F. Mastrogiuseppe, S. Ostojic, and O. Barak, Dynamics of random recurrent networks with correlated low-rank structure, Phys. Rev. Res. 2, 013111 (2020).
  8. A. Valente, J. W. Pillow, and S. Ostojic, Extracting computational mechanisms from neural data using low-rank rnns, Adv. Neural Inf. Process. Syst. 35, 24072 (2022).
  9. L. Cimeša, L. Ciric, and S. Ostojic, Geometry of population activity in spiking networks with low-rank structure, PLoS Comput. Biol. 19, e1011315 (2023).
  10. M. Beiran, N. Meirhaeghe, H. Sohn, M. Jazayeri, and S. Ostojic, Parametric control of flexible timing through low-dimensional neural manifolds, Neuron 111, 739 (2023).
  11. F. Mastrogiuseppe, J. Carmona, and C. K. Machens, Stochastic activity in low-rank recurrent neural networks, PLoS Comput. Biol. 21, e1013371 (2025).
  12. D. G. Clark, O. Marschall, A. van Meegen, and A. Litwin-Kumar, Connectivity structure and dynamics of nonlinear recurrent neural networks, Phys. Rev. X 15, 041019 (2025).
  13. M. T. Kaufman, M. M. Churchland, S. I. Ryu, and K. V. Shenoy, Cortical activity in the null space: Permitting preparation without movement, Nat. Neurosci. 17, 440 (2014).
  14. J. A. Gallego, M. G. Perich, L. E. Miller, and S. A. Solla, Neural manifolds for the control of movement, Neuron 94, 978 (2017).
  15. C. Stringer, M. Pachitariu, N. Steinmetz, C. B. Reddy, M. Carandini, and K. D. Harris, Spontaneous behaviors drive multidimensional, brainwide activity, Science 364, eaav7893 (2019).
  16. C. Stringer, M. Pachitariu, N. Steinmetz, M. Carandini, and K. D. Harris, High-dimensional geometry of population responses in visual cortex, Nature (London) 571, 361 (2019).
  17. F. Lanore, N. A. Cayco-Gajic, H. Gurnani, D. Coyle, and R. A. Silver, Cerebellar granule cell axons support high-dimensional representations, Nat. Neurosci. 24, 1142 (2021).
  18. L. Avitan and C. Stringer, Not so spontaneous: Multi-dimensional representations of behaviors and context in sensory areas, Neuron 110, 3064 (2022).
  19. A. Markanday, S. Hong, J. Inoue, E. De Schutter, and P. Thier, Multidimensional cerebellar computations for flexible kinematic control of movements, Nat. Commun. 14, 2548 (2023).
  20. J. Manley, S. Lu, K. Barber, J. Demas, H. Kim, D. Meyer, F. M. Traub, and A. Vaziri, Simultaneous, cortex-wide dynamics of up to 1 million neurons reveal unbounded scaling of dimensionality with neuron number, Neuron 112, 1694 (2024).
  21. V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, Context-dependent computation by recurrent dynamics in prefrontal cortex, Nature (London) 503, 78 (2013).
  22. D. Sussillo and O. Barak, Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks, Neural Comput. 25, 626 (2013).
  23. C. Huang, D. A. Ruff, R. Pyle, R. Rosenbaum, M. R. Cohen, and B. Doiron, Circuit models of low-dimensional shared variability in cortical networks, Neuron 101, 337 (2019).
  24. D. J. O’Shea, L. Duncker, W. Goo, X. Sun, S. Vyas, E. M. Trautmann, I. Diester, C. Ramakrishnan, K. Deisseroth, M. Sahani et al., Direct neural perturbations reveal a dynamical mechanism for robust computation, bioRxiv 2022 (2022).
  25. F. Schuessler, F. Mastrogiuseppe, S. Ostojic, and O. Barak, Aligned and oblique dynamics in recurrent neural networks, eLife 13, RP93060 (2024).
  26. H. Schwerdtfeger, Introduction to Linear Algebra and the Theory of Matrices (P. Noordhoff, Groningen, 1950).
  27. D. S. Bernstein, Scalar, Vector, and Matrix Mathematics: Theory, Facts, and Formulas-Revised and Expanded Edition (Princeton University Press, Princeton, NJ, 2018).
  28. L. N. Trefethen, Pseudospectra of matrices, Numer. Funct. Anal. Optim. 91, 234 (1991).
  29. B. K. Murphy and K. D. Miller, Balanced amplification: A new mechanism of selective amplification of neural activity patterns, Neuron 61, 635 (2009).
  30. G. Hennequin, T. P. Vogels, and W. Gerstner, Non-normal amplification in random balanced neuronal networks, Phys. Rev. E 86, 011909 (2012).
  31. C. Van Vreeswijk and H. Sompolinsky, Chaos in neuronal networks with balanced excitatory and inhibitory activity, Science 274, 1724 (1996).
  32. I. D. Landau, R. Egger, V. J. Dercksen, M. Oberlaender, and H. Sompolinsky, The impact of structural heterogeneity on excitation-inhibition balance in cortical networks, Neuron 92, 1106 (2016).
  33. R. Pyle and R. Rosenbaum, Highly connected neurons spike less frequently in balanced networks, Phys. Rev. E 93, 040302(R) (2016).
  34. C. Ebsch and R. Rosenbaum, Imbalanced amplification: A mechanism of amplification and suppression from local imbalance of excitation and inhibition in cortical circuits, PLoS Comput. Biol. 14, e1006048 (2018).
  35. V. L. Girko, Circular law, Theory Probab. Appl. 29, 694 (1985).
  36. T. Tao, V. Vu, and M. Krishnapur, Random matrices: Universality of ESDs and the circular law, Ann. Probab. 38, 2023 (2010).
  37. H. Sompolinsky, A. Crisanti, and H.-J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
  38. C. van Vreeswijk and H. Sompolinsky, Chaotic balanced state in a model of cortical circuits, Neural Comput. 10, 1321 (1998).
  39. W. Maass, T. Natschläger, and H. Markram, Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Comput. 14, 2531 (2002).
  40. H. Jaeger and H. Haas, Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication, Science 304, 78 (2004).
  41. A. Renart, J. De La Rocha, P. Bartho, L. Hollender, N. Parga, A. Reyes, and K. D. Harris, The asynchronous state in cortical circuits, Science 327, 587 (2010).
  42. R. Rosenbaum, M. A. Smith, A. Kohn, J. E. Rubin, and B. Doiron, The spatial structure of correlated neuronal variability, Nat. Neurosci. 20, 107 (2017).
  43. C. Baker, C. Ebsch, I. Lampl, and R. Rosenbaum, Correlated states in balanced neuronal networks, Phys. Rev. E 99, 052414 (2019).
  44. C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences, 2nd ed. (Springer-Verlag, Berlin, 1985).
  45. D. Sussillo and L. F. Abbott, Generating coherent patterns of activity from chaotic neural networks, Neuron 63, 544 (2009).
  46. R. Rosenbaum, Modeling Neural Circuits Made Simple with python (MIT Press, Cambridge, MA, 2024).
  47. D. J. MacKay and K. D. Miller, Analysis of Linsker’s application of Hebbian rules to linear networks, Network 1, 257 (1990).
  48. S. Gu, M. G. Mattar, H. Tang, and G. Pan, Emergence and reconfiguration of modular structure for artificial neural networks during continual familiarity detection, Sci. Adv. 10, eadm8430 (2024).
  49. R. B. Levy and A. D. Reyes, Spatial profile of excitatory and inhibitory synaptic connectivity in mouse primary auditory cortex, J. Neurosci. 32, 5609 (2012).
  50. C. K. Pfeffer, M. Xue, M. He, Z. J. Huang, and M. Scanziani, Inhibition of inhibition in visual cortex: The logic of connections between molecularly distinct interneurons, Nat. Neurosci. 16, 1068 (2013).
  51. J. Barral and A. D Reyes, Synaptic scaling rule preserves excitatory–inhibitory balance and salient neuronal network dynamics, Nat. Neurosci. 19, 1690 (2016).
  52. M. Okun and I. Lampl, Instantaneous correlation of excitation and inhibition during ongoing and sensory-evoked activities, Nat. Neurosci. 11, 535 (2008).
  53. B. V. Atallah and M. Scanziani, Instantaneous modulation of gamma oscillation frequency by balancing excitation with inhibition, Neuron 62, 566 (2009).
  54. H. Adesnik and M. Scanziani, Lateral competition for cortical space by layer-specific horizontal circuits, Nature (London) 464, 1155 (2010).
  55. Y. Shu, A. Hasenstaub, and D. A. McCormick, Turning on and off recurrent balanced cortical activity, Nature (London) 423, 288 (2003).
  56. M. Wehr and A. M. Zador, Balanced inhibition underlies tuning and sharpens spike timing in auditory cortex, Nature (London) 426, 442 (2003).
  57. B. Haider, A. Duque, A. R. Hasenstaub, and D. A. McCormick, Neocortical network activity in vivo is generated through a dynamic balance of excitation and inhibition., J. Neurosci. 26, 4535 (2006).
  58. A. L. Dorrn, K. Yuan, A. J. Barker, C. E. Schreiner, and R. C. Froemke, Developmental sensory experience balances cortical excitation and inhibition., Nature (London) 465, 932 (2010).
  59. R. Mastrandrea, J. Fournet, and A. Barrat, Contact patterns in a high school: A comparison between data collected using wearable sensors, contact diaries and friendship surveys, PLoS One 10, e0136497 (2015).
  60. https://github.com/RobertRosenbaum/HighDimLowDimCode.
  61. J. Trousdale, Y. Hu, E. Shea-Brown, and K. Josić, Impact of network structure and cellular response on spike time correlations, PLoS Comput. Biol. 8, e1002408 (2012).
  62. C. Baker, E. Froudarakis, D. Yatsenko, A. S. Tolias, and R. Rosenbaum, Inference of synaptic connectivity and external variability in neural microcircuits, J. Comput. Neurosci. 48, 123 (2020).

Outline

Information

Phys. Rev. X 16, 031025– Published 3 August, 2026
  • Received 12 November 2025
  • Accepted 28 May 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation