- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Ghost Mechanism: An Analytical Model of Abrupt Learning in Recurrent Networks

Fatih Dinc1,2,3,4,*, Ege Cirakman3,*, Bariscan Kurtkaya3,5,*, Mert Yuksekgonul6, Yiqi Jiang3, Mark J. Schnitzer3,7,8,†, and Hidenori Tanaka4,9,†

  • *Principal author.
  • †These authors contributed equally to this work.
PDF

Phys. Rev. X 16, 021058 – Published 23 June, 2026

DOI: https://doi.org/10.1103/mjcl-lb4x

Abstract

Abrupt learning, long performance plateaus followed by rapid convergence, is a common phenomenon in recurrent neural networks (RNNs) trained on working-memory tasks. In such cases, the networks develop transient slow regions in state space that extend the effective timescales of computation. However, the mechanisms driving sudden performance improvements and their causal role remain unclear, largely because we lack an analytical dynamical-systems framework. To address this gap, we introduce the ghost mechanism, a general process by which finite-dimensional continuous-time dynamical systems exhibit transient slowdown near the remnant of a saddle-node bifurcation. By reducing the high-dimensional dynamics near ghost points, we derive a one-dimensional canonical form that analytically captures learning as a process controlled by a single scale parameter. Using this model, we study a form of abrupt learning emerging from ghost points and identify a critical learning rate that scales as an inverse power law with the timescale of the learned computation. Beyond this rate, learning collapses through two interacting modes: (i) vanishing gradients and (ii) oscillatory gradients near minima. These features can lock the system into high confidence but incorrect predictions when parameter updates trigger a no-learning zone, a region of parameter space where gradients vanish. We validate these predictions in low-rank RNNs, where ghost points precede abrupt transitions and further demonstrate their generality in full-rank RNNs trained on canonical working-memory tasks. Our theory offers two approaches to address these learning difficulties: Increasing trainable ranks stabilizes learning trajectories, while reducing output confidence mitigates entrapment in no-learning zones. Overall, the ghost mechanism reveals how the computational demands of a task constrain the optimization landscape, demonstrating that well-known learning difficulties in RNNs partly arise from the dynamical systems they must learn to implement.

View figure in article

Physics Subject Headings (PhySH)

Popular Summary

Neural networks often learn in bursts: performance can remain flat for long periods before suddenly improving. This pattern is common when recurrent neural networks are trained to hold information in memory, yet a mechanistic explanation has been lacking. In this work, we identify one concrete mechanism that can give rise to such abrupt learning. When networks learn to temporarily store information, their dynamics can develop ghosts: transient bottlenecks that slow neural activity and act as delayed switches. By studying a canonical model describing ghost formation, we discovered that learning longer delays in switching requires ever-smaller learning steps. Train too fast, and the network gets stuck in a no-learning zone where training grinds to a halt. We found two practical fixes: high-dimensional dynamics and less-confident outputs help avoid training pitfalls. Our results reveal how the dynamical mechanisms that emerge during training can themselves constrain learning in recurrent networks.

Article Text

References (96)

  1. Donald Olding Hebb, The Organization of Behavior: A Neuropsychological Theory (Psychology Press, New York, 2005).
  2. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell, Steven Siegelbaum, A. James Hudspeth, Sarah Mack et al., Principles of Neural Science (McGraw-Hill New York, 2000), Vol. 4.
  3. Anne E. Urai, Brent Doiron, Andrew M. Leifer, and Anne K. Churchland, Large-scale neural recordings call for new insights to link brain and behavior, Nat. Neurosci. 25, 11 (2022).
  4. Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein, Tianyu Wang, Darren T. Schachter, Zoey Hu, and Peter L. McMahon, Deep physical neural networks trained with backpropagation, Nature (London) 601, 549 (2022).
  5. Peter L. McMahon, The physics of optical computing, Nat. Rev. Phys. 5, 717 (2023).
  6. Sunil Pai, Zhanghao Sun, Tyler W. Hughes, Taewon Park, Ben Bartlett, Ian AD Williamson, Momchil Minkov, Maziyar Milanizadeh, Nathnael Abebe, Francesco Morichetti et al., Experimentally realized in situ backpropagation for deep learning in photonic neural networks, Science 380, 398 (2023).
  7. Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick, Neuroscience-inspired artificial intelligence, Neuron 95, 245 (2017).
  8. Anthony M. Zador, A critique of pure learning and what artificial neural networks can learn from animal brains, Nat. Commun. 10, 3770 (2019).
  9. Lukas Eisenmann, Zahra Monfared, Niclas Göring, and Daniel Durstewitz, Bifurcations and loss jumps in RNN training, Adv. Neural Inf. Process. Syst. 36, 70511 (2023).
  10. Udith Haputhanthri, Liam Storan, Yiqi Jiang, Adam Shai, Hakki Orhun Akengin, Mark Schnitzer, Fatih Dinc, and Hidenori Tanaka, Why do recurrent neural networks suddenly learn? Bifurcation mechanisms in neuro-inspired short-term-memory tasks, in ICML 2024 Workshop on Mechanistic Interpretability (ICML 2024 Workshop on Mechanistic Interpretability, Vienna, Austria, 2024).
  11. Peter DelMastro, Rushiv Arora, Edward Rietman, and Hava T. Siegelmann, On the dynamics of learning time-aware behavior with recurrent neural networks, arXiv:2306.07125.
  12. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso et al., Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, arXiv:2206.04615.
  13. Sanjeev Arora and Anirudh Goyal, A theory for emergence of complex skills in language models, arXiv:2307.15936.
  14. Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, and Sanjeev Arora, Skill-mix: A flexible and expandable family of evaluations for AI models, arXiv:2310.17567.
  15. Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P. Dick, and Hidenori Tanaka, A percolation model of emergence: Analyzing transformers trained on a formal language, arXiv:2408.12578.
  16. Maya Okawa, Ekdeep S. Lubana, Robert Dick, and Hidenori Tanaka, Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task, Adv. Neural Inf. Process. Syst. 36, 50173 (2023).
  17. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler et al., Emergent abilities of large language models, arXiv:2206.07682.
  18. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark et al., Training compute-optimal large language models, Adv. Neural Inf. Process. Syst. 35, 30016 (2022).
  19. Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra, Grokking: Generalization beyond overfitting on small algorithmic datasets, arXiv:2201.02177.
  20. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio, On the difficulty of training recurrent neural networks, in International Conference on Machine Learning (PMLR, 2013), pp. 1310–1318, https://dl.acm.org/doi/10.5555/3042817.3043083.
  21. Kenji Doya et al., Bifurcations in the learning of recurrent neural networks 3, Learning (RTRL) 3, 17 (1992).
  22. Ian J. Goodfellow, Oriol Vinyals, and Andrew M. Saxe, Qualitatively characterizing neural network optimization problems, arXiv:1412.6544.
  23. Gautam Reddy, The mechanistic basis of data dependence and abrupt learning in an in-context classification task, In The Twelfth International Conference on Learning Representations (ICLR, Vienna, Austria, 2023).
  24. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen et al., In-context learning and induction heads, arXiv:2209.11895.
  25. David Sussillo and Larry F. Abbott, Generating coherent patterns of activity from chaotic neural networks, Neuron 63, 544 (2009).
  26. Florian Hess, Zahra Monfared, Manuel Brenner, and Daniel Durstewitz, Generalized teacher forcing for learning chaotic dynamics, in Proceedings of the 40th International Conference on Machine Learning, ICML’23 (2023), https://dl.acm.org/doi/10.5555/3618408.3618937.
  27. Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, and Guy Gur-Ari, The large learning rate phase of deep learning: The catapult mechanism, arXiv:2003.02218.
  28. Yasaman Bahri, Jonathan Kadmon, Jeffrey Pennington, Sam S. Schoenholz, Jascha Sohl-Dickstein, and Surya Ganguli, Statistical mechanics of deep learning, Annu. Rev. Condens. Matter Phys. 11, 501 (2020).
  29. Sara Kangaslahti, Elan Rosenfeld, and Naomi Saphra, Loss in the crowd: Hidden breakthroughs in language model training, In ICML 2024 Workshop on Mechanistic Interpretability (ICML, Vienna, Austria, 2024).
  30. Steven H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering (CRC Press, Boca Raton, 2018).
  31. Anton Maximilian Schäfer and Hans Georg Zimmermann, Recurrent neural networks are universal approximators, in International Conference on Artificial Neural Networks (Springer, New York, 2006), pp. 632–640, 10.1007/11840817_66.
  32. Jack Carr, Applications of Centre Manifold Theory (Springer Science & Business Media, New York, 2012), Vol. 35.

  33. Bifurcations are formally defined as shifts in system behavior, which alter the number or stability of fixed points, periodic orbits, or other invariant sets [26, 30, 34, 35, 36, 37, 38, 39].

  34. Dominik Schmidt, Georgia Koppe, Zahra Monfared, Max Beutelspacher, and Daniel Durstewitz, Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies, In International Conference on Learning Representations (2021).
  35. Robert Haschke and Jochen J. Steil, Input space bifurcation manifolds of recurrent neural networks, Neurocomputing;Variable Star Bulletin 64, 25 (2005).
  36. Alexander Rehmer and Andreas Kroll, The effect of the forget gate on bifurcation boundaries and dynamics in recurrent neural networks and its implications for gradient-based optimization, In 2022 International Joint Conference on Neural Networks (IJCNN) (IEEE, Padua, Italy, 2022), pp. 01–08, 10.1109/IJCNN55064.2022.9892458.
  37. Viktor Avrutin, Michael Schanz, and Soumitro Banerjee, Occurrence of multiple attractor bifurcations in the two-dimensional piecewise linear normal form map, Nonlinear Dyn. 67, 293 (2012).
  38. A. Ganguli and S. Banerjee, Dangerous bifurcation at border collision: When does it occur?, Phys. Rev. E 71, 057202 (2005).
  39. Z. Monfared and D. Durstewitz, Existence of n-cycles and border-collision bifurcations in piecewise-linear continuous maps with applications to recurrent neural networks, Nonlinear Dyn. 101, 1037 (2020).
  40. Yoshua Bengio, Patrice Simard, and Paolo Frasconi, Learning long-term dependencies with gradient descent is difficult, IEEE Trans. Neural Networks 5, 157 (1994).
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin, Attention is all you need, Adv. Neural Inf. Process. Syst. 30, 6000 (2017), https://dl.acm.org/doi/10.5555/3295222.3295349.

  42. As a side note for precision, for our derivations in this section, we assume that f[x(t)] is a sufficiently smooth vector field that can be Taylor expanded to at least second order, with a nonvanishing quadratic coefficient along the slow-mode direction. The exact assumption will be clear at the end of the calculation.

  43. Yuri A. Kuznetsov, Elements of Applied Bifurcation Theory (Springer, New York, 1998).

  44. To accommodate slow-point manifolds and not just isolated points, one could rework this definition. For our purposes, the distinction is not relevant for the canonical model.

  45. David Sussillo and Omri Barak, Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks, Neural Comput. 25, 626 (2013).
  46. S. H. Strogatz and R. M. Westervelt, Predicted power laws for delayed switching of charge-density waves, Phys. Rev. B 40, 10501 (1989).
  47. D. Koch, A. Nandan, G. Ramesan, I. Tyukin, A. Gorban, and A. Koseska, Ghost channels and ghost cycles guiding long transients in dynamical systems, Phys. Rev. Lett. 133, 047202 (2024).
  48. Diederik P. Kingma and Jimmy Ba, Adam: A method for stochastic optimization, arXiv:1412.6980.
  49. Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht, The marginal value of adaptive gradient methods in machine learning, Adv. Neural Inf. Process. Syst. 30, 4151 (2017), https://dl.acm.org/doi/10.5555/3294996.3295170.
  50. Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein, Visualizing the loss landscape of neural nets, Adv. Neural Inf. Process. Syst. 31 (2018), https://dl.acm.org/doi/10.5555/3327345.3327535.
  51. Adrian Valente, Jonathan W. Pillow, and Srdjan Ostojic, Extracting computational mechanisms from neural data using low-rank rnns, Adv. Neural Inf. Process. Syst. 35, 24072 (2022).
  52. Alexis Dubreuil, Adrian Valente, Manuel Beiran, Francesca Mastrogiuseppe, and Srdjan Ostojic, The role of population structure in computations through neural dynamics, Nat. Neurosci. 25, 783 (2022).
  53. Fatih Dinc, Marta Blanco-Pozo, David Klindt, Francisco Acosta, Yiqi Jiang, Sadegh Ebrahimi, Adam Shai, Hidenori Tanaka, Peng Yuan, Mark J. Schnitzer, and Nina Miolane, Latent computing by biological neural networks: A dynamical systems framework, arXiv:2502.14337.
  54. Manuel Beiran, Alexis Dubreuil, Adrian Valente, Francesca Mastrogiuseppe, and Srdjan Ostojic, Shaping dynamics with multiple populations in low-rank recurrent networks, Neural Comput. 33, 1572 (2021).
  55. Francesca Mastrogiuseppe and Srdjan Ostojic, Linking connectivity, dynamics, and computations in low-rank recurrent neural networks, Neuron 99, 609 (2018).
  56. F. Dinc, Code for “Ghost Mechanism: An Analytical Model of Abrupt Learning in Recurrent Networks”. Zenodo, (2024), 10.5281/zenodo.13689162.
  57. A. J. Bray and D. S. Dean, Statistics of critical points of Gaussian fields on large-dimensional spaces, Phys. Rev. Lett. 98, 150201 (2007).
  58. Nicolas Y. Masse, Guangyu R. Yang, H. Francis Song, Xiao-Jing Wang, and David J. Freedman, Circuit mechanisms for the maintenance and manipulation of information in working memory, Nat. Neurosci. 22, 1159 (2019).
  59. Guangyu Robert Yang, Madhura R. Joglekar, H. Francis Song, William T. Newsome, and Xiao-Jing Wang, Task representations in neural networks trained to perform many cognitive tasks, Nat. Neurosci. 22, 297 (2019).
  60. Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo, Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics, Adv. Neural Inf. Process. Syst. 32, 15696 (2019), https://dl.acm.org/doi/10.5555/3454287.3455693.
  61. Antônio H. Ribeiro, Koen Tiels, Luis A. Aguirre, and Thomas Schön, Beyond exploding and vanishing gradients: Analysing rnn training using attractors and smoothness, in International Conference on Artificial Intelligence and Statistics, Palermo, Sicily, Italy. (PMLR, 2020), pp. 2370–2380, https://proceedings.mlr.press/v108/ribeiro20a.html.
  62. Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo, Universality and individuality in neural dynamics across large populations of recurrent networks, Adv. Neural Inf. Process. Syst. 32, 15629 (2019), https://dl.acm.org/doi/10.5555/3454287.3455687.
  63. Xavier Glorot and Yoshua Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, JMLR Workshop and Conference Proceedings (PMLR, 2010), pp. 249–256, https://proceedings.mlr.press/v9/glorot10a.html.
  64. Xavier Glorot, Antoine Bordes, and Yoshua Bengio, Deep sparse rectifier neural networks, in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, JMLR Workshop and Conference Proceedings (PMLR, 2011), pp. 315–323, https://proceedings.mlr.press/v15/glorot11a.html.

  65. That is, unless initialization occurs in a parameter regime, which in full-rank RNNs leads to chaos [66]. Studying an equivalent initialization regime in low-rank RNNs constitutes an entirely different research direction.

  66. H. Sompolinsky, A. Crisanti, and H. J. Sommers, Chaos in random neural networks, Phys. Rev. Lett. 61, 259 (1988).
  67. Blake A. Richards, Timothy P. Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli et al., A deep learning framework for neuroscience, Nat. Neurosci. 22, 1761 (2019).
  68. Valerio Mante, David Sussillo, Krishna V. Shenoy, and William T. Newsome, Context-dependent computation by recurrent dynamics in prefrontal cortex, Nature (London) 503, 78 (2013).
  69. Matthew G. Perich, Charlotte Arlt, Sofia Soares, Megan E. Young, Clayton P. Mosher, Juri Minxha, Eugene Carter, Ueli Rutishauser, Peter H. Rudebeck, Christopher D. Harvey et al., Inferring brain-wide interactions using data-constrained recurrent neural network models, bioRxiv (2021), pp. 2020–12, 10.1101/2020.12.18.423348.
  70. Matthew G. Perich, Devika Narain, and Juan A. Gallego, A neural manifold view of the brain, Nat. Neurosci. 28, 1582 (2025).
  71. Matthew G. Perich and Kanaka Rajan, Rethinking brain-wide interactions through multi-region ‘network of networks’ models, Current Opinion Neurobiol. 65, 146 (2020).
  72. Mengyu Liu, Aditya Nair, Nestor Coria, Scott W. Linderman, and David J. Anderson, Encoding of female mating dynamics by a hypothalamic line attractor, Nature (London) 634, 901 (2024).
  73. Abhranil Das and Ila R. Fiete, Systematic errors in connectivity inferred from activity in strongly recurrent networks, Nat. Neurosci. 23, 1286 (2020).
  74. Kiyohito Iigaya, Madalena S. Fonseca, Masayoshi Murakami, Zachary F. Mainen, and Peter Dayan, An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals, Nat. Commun. 9, 2477 (2018).
  75. Cooper D. Grossman, Bilal A. Bari, and Jeremiah Y. Cohen, Serotonin neurons modulate learning rate through uncertainty, Curr. Biol. 32, 586 (2022).
  76. Luke T. Coddington, Sarah E. Lindo, and Joshua T. Dudman, Mesolimbic dopamine adapts the rate of learning from action, Nature (London) 614, 294 (2023).
  77. Kanaka Rajan, Christopher D. Harvey, and David W. Tank, Recurrent network models of sequence generation and memory, Neuron 90, 128 (2016).
  78. Joaquin M. Fuster and Garrett E. Alexander, Neuron activity related to short-term memory, Science 173, 652 (1971).
  79. Saurabh Vyas, Matthew D. Golub, David Sussillo, and Krishna V. Shenoy, Computation through neural population dynamics, Annu. Rev. Neurosci. 43, 249 (2020).
  80. Christopher J. Honey, Thomas Thesen, Tobias H. Donner, Lauren J. Silbert, Chad E. Carlson, Orrin Devinsky, Werner K. Doyle, Nava Rubin, David J. Heeger, and Uri Hasson, Slow cortical dynamics and the accumulation of information over long timescales, Neuron 76, 423 (2012).
  81. Fatih Dinc, Adam Shai, Mark Schnitzer, and Hidenori Tanaka, Cornn: Convex optimization of recurrent neural networks for rapid inference of neural dynamics, Adv. Neural Inf. Process. Syst. 36, 51273 (2023).
  82. Ábel Ságodi, Guillermo Martín-Sánchez, Piotr Sokol, and Memming Park, Back to the continuous attractor, Adv. Neural Inf. Process. Syst. 37, 66856 (2024).
  83. Bariscan Kurtkaya, Fatih Dinc, Mert Yuksekgonul, Marta Blanco-Pozo, Ege Cirakman, Mark J. Schnitzer, Yucel Yemez, Hidenori Tanaka, Peng Yuan, and Nina Miolane, Dynamical phases of short-term memory mechanisms in RNNS, in Proceedings of the 42nd International Conference on Machine Learning, ICML’25 (2025), https://dl.acm.org/doi/10.5555/3780338.3781597.
  84. Renate Krause, Matthew Cook, Sepp Kollmorgen, Valerio Mante, and Giacomo Indiveri, Operative dimensions in unconstrained connectivity of recurrent neural networks, Adv. Neural Inf. Process. Syst. 35, 17073 (2022).
  85. Ann Huang, Satpreet H. Singh, Flavio Martinelli, and Kanaka Rajan, Measuring and controlling solution degeneracy across task-trained recurrent neural networks, arXiv:2410.03972.
  86. Friedrich Schuessler, Francesca Mastrogiuseppe, Alexis Dubreuil, Srdjan Ostojic, and Omri Barak, The interplay between randomness and structure during learning in RNNs, Adv. Neural Inf. Process. Syst. 33, 13352 (2020), https://dl.acm.org/doi/abs/10.5555/3495724.3496844.
  87. Laura Balzano, Tianjiao Ding, Benjamin D. Haeffele, Soo Min Kwon, Qing Qu, Peng Wang, Zhangyang Wang, and Can Yaras, An overview of low-rank structures in the training and adaptation of large models, arXiv:2503.19859.
  88. Albert Gu and Tri Dao, Mamba: Linear-time sequence modeling with selective state spaces, in First Conference on Language Modeling, Philadelphia, PA, USA (COLM, 2024).
  89. Christian Klos and Raoul-Martin Memmesheimer, Smooth exact gradient descent learning in spiking neural networks, Phys. Rev. Lett. 134, 027301 (2025).
  90. Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass, A solution to the learning dilemma for recurrent networks of spiking neurons, Nat. Commun. 11, 3625 (2020).
  91. Michael Pfeiffer and Thomas Pfeil, Deep learning with spiking neurons: Opportunities and challenges, Front. Neurosci. 12, 409662 (2018).
  92. Zhaoyang Niu, Guoqiang Zhong, and Hui Yu, A review on the attention mechanism of deep learning, Neurocomputing;Variable Star Bulletin 452, 48 (2021).
  93. Ryan N. Gutenkunst, Joshua J. Waterfall, Fergal P. Casey, Kevin S. Brown, Christopher R. Myers, and James P. Sethna, Universally sloppy parameter sensitivities in systems biology models, PLoS Comput. Biol. 3, e189 (2007).
  94. Scott Kirkpatrick, C. Daniel Gelatt Jr, and Mario P. Vecchi, Optimization by simulated annealing, Science 220, 671 (1983).
  95. Neal A. Hessler and Allison J. Doupe, Social context modulates singing-related neural activity in the songbird forebrain, Nat. Neurosci. 2, 209 (1999).
  96. Mimi H. Kao, Brian D. Wright, and Allison J. Doupe, Neurons in a forebrain nucleus required for vocal plasticity rapidly switch between precise firing and variable bursting depending on social context, J. Neurosci. 28, 13232 (2008).

Outline

Information

Phys. Rev. X 16, 021058– Published 23 June, 2026
  • Received 9 January 2025
  • Revised 17 May 2025
  • Accepted 8 April 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation