- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Taxonomy of prediction

Alexandra Jurgens1,*,† and James P. Crutchfield2,‡,§

PDF

Phys. Rev. Research 8, 023218 – Published 28 May, 2026

DOI: https://doi.org/10.1103/4cg9-fzpp

Abstract

A prediction makes a claim about a system's future given knowledge of its past. A retrodiction makes a claim about its past given knowledge of its future. The bidirectional machine is an ambidextrous hidden Markov chain that does both optimally by making explicit in its state structure all statistical correlations in a stochastic process. We introduce an informational taxonomy to profile these correlations via a suite of multivariate information measures. While prior results laid out the different kinds of information contained in isolated measurements, the associated informations were challenging to calculate explicitly. Overcoming this via bidirectional machine states, we expand that analysis to prediction and retrodiction. The result highlights fourteen interpretable and calculable measures that characterize a process's informational structure. In addition, we introduce a labeling and indexing scheme that systematizes information-theoretic analyses of complex multivariate systems. Operationalizing this, we provide algorithms to directly calculate all of these quantities in closed form for finitely modeled processes.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (32)

  1. C. E. Shannon, A mathematical theory of communication, Bell Sys. Tech. J. 27, 379 (1948).
  2. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. (Wiley-Interscience, New York, 2006).
  3. R. W. Yeung, Information Theory and Network Coding (Springer, New York, 2008).
  4. J. Bernoulli, Ars Conjectandi, Opus posthumum, accedit tractatus de seriebus infinitis, et epistola Gallice scripta de ludo pilae recticularis, Basileae, 1713, chapters 1–4 translated into English by B. Sung, Ars Conjectandi, Technical Report No. 2, (Department of Statistics, Harvard University, 1966).
  5. S.-D. Poisson, Recherches sur la Probabilité des Jugements en Matiere Criminelle et en Metière Civile, Préédées des Regles Généales du Calcul des Probabilitiés, Imprimeur–Libraire pour les Mathematiques (Bachelier, Paris, 1837).
  6. P. L. Chebyshev, Des valuers moyennes, J. Math. Pure Appl. 12, 177 (1867).
  7. A. A. Markov, Rasprostranenie predel’nyh teorem ischisleniya veroyatnostej na summu velichin svyazannyh v cep’, Zapiski Akademii Nauk po Fiziko-Matematicheskomu Otdeleniyu 3, (1908). German translation: Ausdehnung der Satze uber die Grenzwerte in der Wahrscheinlichkeitsrechnung auf eine Summe verketteter Grossen, in Wahrscheinlichkeitsrechnung, edited by A. A. Markoff, translated by H. Liebmann (B. G. Teuber, Leipzig, 1912), pp. 272–298. English translation: Extension of the limit theorems of probability theory to a sum of variables connected in a chain, translated by S. Petelin, in Dynamic Probabilities Systems, edited by R. A. Howard (Wiley, New York, 1971), Vol. 1, pp. 552–576.
  8. C. E. Shannon, Prediction and entropy of printed English, Bell Sys. Tech. J. 30, 50 (1951).
  9. R. G. James, C. J. Ellison, and J. P. Crutchfield, Anatomy of a bit: Information in a time series observation, CHAOS 21, 037109 (2011).
  10. C. R. Shalizi and J. P. Crutchfield, Computational mechanics: Pattern and prediction, structure and simplicity, J. Stat. Phys. 104, 817 (2001).
  11. J. P. Crutchfield, C. J. Ellison, and J. R. Mahoney, Time's barbed arrow: Irreversibility, crypticity, and stored information, Phys. Rev. Lett. 103, 094101 (2009).
  12. O. Kallenberg, Foundations of Modern Probability, 2nd ed. (Springer, New York, 2001).
  13. P. Kůrka, Topological and Symbolic Dynamics (Société Mathématique de France, Paris, 2003).
  14. H. K. Ting, On the amount of information, Theory Probab. Appl. 7, 439 (1962).
  15. R. G. James and J. P. Crutchfield, Multivariate dependence beyond Shannon information, Entropy 19, 531 (2017).
  16. R. D. Beer and P. L. Williams, Information processing and dynamics in minimally cognitive agents, Cognit. Sci. 39, 1 (2015).
  17. C. J. Ellison, J. R. Mahoney, and J. P. Crutchfield, Prediction, retrodiction, and the amount of information stored in the present, J. Stat. Phys. 136, 1005 (2009).
  18. R. G. James, K. Burke, and J. P. Crutchfield, Chaos forgets and remembers: Measuring information creation, destruction, and storage, Phys. Lett. A 378, 2124 (2014).
  19. A. Jurgens and J. P. Crutchfield, Divergent predictive states: The statistical complexity dimension of stationary, ergodic hidden Markov processes, Chaos 31, 083114 (2021).
  20. B. D. Johnson, J. P. Crutchfield, C. J. Ellison, and C. S. McTague, Enumerating finitary processes, Entropy 26, 1105 (2024).
  21. J. P. Crutchfield, P. Riechers, and C. J. Ellison, Exact complexity: Spectral decomposition of intrinsic computation, Phys. Lett. A 380, 998 (2016).
  22. C. C. Strelioff and J. P. Crutchfield, Bayesian structural inference for hidden processes, Phys. Rev. E 89, 042119 (2014).
  23. S. E. Marzen and J. P. Crutchfield, Inference, prediction, and entropy-rate estimation of continuous-time, discrete-event processes, Entropy 24, 1675 (2022).
  24. C. R. Shalizi and K. L. Klinkner, Blind construction of optimal nonlinear recursive predictors for discrete sequences, in Uncertainty in Artificial Intelligence: Proceedings of the Twentieth Conference, edited by M. Chickering and J. Y. Halpern (AUAI Press, Arlington, Virginia, 2004), pp. 504–511.
  25. N. Brodu and J. P. Crutchfield, Discovering causal structure with reproducing-kernel Hilbert space ε-machines, Chaos 32, 023103 (2022).
  26. A. M. Jurgens and N. Brodu, Inferring kernel ε-machines: Discovering structure in complex systems, Chaos 35, 033162 (2025).
  27. J. P. Crutchfield and C. Aghamohammdi, Not all fluctuations are created equal: Spontaneous variations in thermodynamic function, Entropy 26, 894 (2024).
  28. P. L. Williams and R. D. Beer, Nonnegative decomposition of multivariate information, https://arxiv.org/abs/1004.2515.
  29. L. Faes, L. Sparacino, G. Mijatovic, Y. Antonacci, L. Ricci, D. Marinazzo, and S. Stramaglia, Partial information rate decomposition, Phys. Rev. Lett. 135, 187401 (2025).
  30. S. Watanabe, Information theoretical analysis of multivariate correlation, IBM J. Res. Dev. 4, 66 (1960).
  31. T. S. Han, Linear dependence structure of the entropy space, Inf. Control 29, 337 (1975).
  32. P. Gács and J. Körner, Common information is much less than mutual information, Probl. Control Inf. Theory 2, 149 (1973).

Outline

Information

Phys. Rev. Research 8, 023218– Published 28 May, 2026
  • Received 7 July 2025
  • Accepted 25 February 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation