• Accepted Paper

Innovation capacity of dynamical learning systems

Anthony M. Polloreno

Phys. Rev. Applied - Accepted 5 November, 2026

DOI: https://doi.org/10.1103/drvs-8fh2

Abstract

In noisy physical reservoirs, the classical information-processing capacity Cip quantifies how well a linear readout can realize tasks measurable from the input history, yet Cip can be far smaller than the observed rank of the readout covariance. We explain this "missing capacity" by introducing the innovation capacity Ci, the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law Cip+Ci=rank(ΣXX)≤d, so predictable and innovation capacities exactly partition the rank of the observable readout covariance ΣXX∈ℝd×d. In linear-Gaussian Johnson-Nyquist regimes, ΣXX(T)=S+TN0, the split becomes a generalized-eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making Ci a trace-controlled innovation budget. A large Ci forces a high-dimensional innovation subspace with a variance floor, and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.

Export citation

Export citation

Choose format for download:

Download Citation

If the author has provided any supplemental materials with this article they will be available upon publication of the version of record.

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation