- Letter
- Open Access
Disentangling high-order effects in the transfer entropy
Phys. Rev. Research 6, L032007 – Published 9 July, 2024
DOI: https://doi.org/10.1103/PhysRevResearch.6.L032007
Abstract
Transfer entropy (TE), the primary method for determining directed information flow within a network system, can exhibit bias—either in deficiency or excess—during both pairwise and conditioned calculations, owing to high-order dependencies among the dynamic processes under consideration and the remaining processes in the system used for conditioning. Here, we propose a novel approach. Instead of conditioning TE on all network processes except the driver and the target, as in its fully conditioned version, or not conditioning at all, as in the pairwise approach, our method searches for both the multiplets of variables that maximize information flow and those that minimize it. This provides a decomposition of TE into unique, redundant, and synergistic atoms. Our approach enables the quantification of the relative importance of high-order effects compared to pure two-body effects in information transfer between two processes, while also highlighting the processes that contribute to building these high-order effects alongside the driver. We demonstrate the application of our approach in climatology by analyzing data from El Niño and the Southern Oscillation.
Physics Subject Headings (PhySH)
Article Text
References (26)
- C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica 37, 424 (1969).
- T. Schreiber, Measuring information transfer, Phys. Rev. Lett. 85, 461 (2000).
- T. Bossomaier, L. Barnett, M. Harré, and J. T. Lizier, An Introduction to Tranfer Entropy (Springer, Berlin, 2016).
- L. Barnett, A. B. Barrett, and A. K. Seth, Granger causality and transfer entropy are equivalent for Gaussian variables, Phys. Rev. Lett. 103, 238701 (2009).
- M. Staniek and K. Lehnertz, Symbolic transfer entropy, Phys. Rev. Lett. 100, 158101 (2008).
- A. B. Barrett and L. Barnett, Granger causality is designed to measure effect, not mechanism, Front. Neuroinf. 7, 6 (2013).
- F. Rosas, P. Mediano, A. Luppi, T. Varley, J. Lizier, S. Stramaglia, H. Jensen, and D. Marinazzo, Disentangling high-order mechanisms and high-order behaviours in complex systems, Nat. Phys. 18, 476 (2022).
- F. Battiston, E. Amico, A. Barrat, G. Bianconi, G. Ferraz, Arruda, B. Franceschiello, I. Iacopini, S. Kéfi, V. Latora, Y. Moreno, M. Murray, T. Peixoto, F. Vaccarino, and G. Petri, The physics of higher-order interactions in complex systems, Nat. Phys. 17, 1093 (2021).
- P. L. Williams and R. D. Beer, Nonnegative decomposition of multivariate information, arXiv:1004.2515.
- J. T. Lizier, N. Bertschinger, J. Jost, and M. Wibral, Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work, Entropy 20, 307 (2018).
- L. Faes, D. Marinazzo, and S. Stramaglia, Multiscale information decomposition: Exact computation for multivariate Gaussian processes, Entropy 19, 408 (2017).
- R. G. James, N. Barnett, and J. P. Crutchfield, Information flows? A critique of transfer entropies, Phys. Rev. Lett. 116, 238701 (2016).
- S. Stramaglia, J. M. Cortes, and D. Marinazzo, Synergy and redundancy in the Granger causal analysis of dynamical networks, New J. Phys. 16, 105003 (2014).
- D. Marinazzo, M. Pellicoro, and S. Stramaglia, Causal information approach to partial conditioning in multivariate data sets, Comput. Math. Methods Med. 2012, 303601 (2012).
- F. E. Rosas, P. A. M. Mediano, M. Gastpar, and H. J. Jensen, Quantifying high-order interdependencies via multivariate extensions of the mutual information, Phys. Rev. E 100, 032305 (2019).
- G. Mijatovic, L. Sparacino, Y. Antonacci, M. Javorka, D. Marinazzo, S. Stramaglia, and L. Faes, Assessing high-order links in cardiovascular and respiratory networks via static and dynamic information measures, IEEE Open J. Eng. Med. Biol. 1, 1 (2024).
- T. Schreiber and A. Schmitz, Improved surrogate data for nonlinearity tests, Phys. Rev. Lett. 77, 635 (1996).
- M. J. McPhaden, S. E. Zebiak, and M. H. Glantz, ENSO as an integrating concept in earth science, Science 314, 1740 (2006).
- R. Silini, G. Tirabassi, M. Barreiro, L. Ferranti, and C. Masoller, Assessing causal dependencies in climatic indices, Clim. Dyn. 61, 79 (2023).
- M. H. Hansen and B. Yu, Model selection and the principle of minimum description length, J. Am. Stat. Assoc. 96, 746 (2001).
- S. Stramaglia, T. Scagliarini, B. C. Daniels, and D. Marinazzo, Quantifying dynamical high-order interdependencies from the O-information: An application to neural spiking dynamics, Front. Physiol. 11, 595736 (2021).
- GitHub, https://github.com/danielemarinazzo/TE_decomposition.
- NOAA website, https://psl.noaa.gov/data/climateindices/list/).
- Indiana Institute of Tropical Meteorology, https://www.tropmet.res.in/.
- W. J. McGill, Multivariate information transmission, Psychometrika 19, 97 (1954).
- A. B. Barrett, Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems, Phys. Rev. E 91, 052802 (2015).
