- Accepted Paper
Innovation capacity of dynamical learning systems
Phys. Rev. Applied - Accepted 5 November, 2026
DOI: https://doi.org/10.1103/drvs-8fh2
Phys. Rev. Applied - Accepted 5 November, 2026
DOI: https://doi.org/10.1103/drvs-8fh2
In noisy physical reservoirs, the classical information-processing capacity quantifies how well a linear readout can realize tasks measurable from the input history, yet can be far smaller than the observed rank of the readout covariance. We explain this "missing capacity" by introducing the innovation capacity , the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law , so predictable and innovation capacities exactly partition the rank of the observable readout covariance . In linear-Gaussian Johnson-Nyquist regimes, , the split becomes a generalized-eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making a trace-controlled innovation budget. A large forces a high-dimensional innovation subspace with a variance floor, and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.
If the author has provided any supplemental materials with this article they will be available upon publication of the version of record.