Criticality has been proposed as a key organizing principle in biological neural systems, yet its origin and relevance in artificial neural networks remain unclear. We analyze hidden-state dynamics in trained long short-term memory (LSTM) networks and show that small networks near their optimal training epochs (steps) exhibit scale-free avalanche statistics and branching parameters close to unity, indicative of near-critical dynamics, while larger models remain subcritical. To explain the coexistence of subcritical branching with robust $1/f^{\beta }$ noise, we introduce a mixture branching process framework that links heterogeneous branching dynamics to long-range temporal correlations. These results identify critical-like behavior in LSTM networks as an emergent, capacity-dependent dynamical regime.

Close

PrevNext

View figure in article

• Critical phenomena
• Neuronal dynamics
• Order parameters
• Patterns in complex systems
• Scaling laws of complex systems
• Self-organized criticality
• Artificial neural networks
• Multiple time scale dynamics
• Stochastic dynamical systems
• Finite-size scaling