Quantifying information transfer and mediation along causal pathways in complex systems

Measures of information transfer have become a popular approach to analyze interactions in complex systems such as the Earth or the human brain from measured time series. Recent work has focused on causal definitions of information transfer aimed at decompositions of predictive information about a target variable, while excluding effects of common drivers and indirect influences. While common drivers clearly constitute a spurious causality, the aim of the present article is to develop measures quantifying different notions of the strength of information transfer along indirect causal paths, based on first reconstructing the multivariate causal network. Another class of novel measures quantifies to what extent different intermediate processes on causal paths contribute to an interaction mechanism to determine pathways of causal information transfer. The proposed framework complements predictive decomposition schemes by focusing more on the interaction mechanism between multiple processes. A rigorous mathematical framework allows for a clear information-theoretic interpretation that can also be related to the underlying dynamics as proven for certain classes of processes. Generally, however, estimates of information transfer remain hard to interpret for nonlinearly intertwined complex systems. But if experiments or mathematical models are not available, then measuring pathways of information transfer within the causal dependency structure allows at least for an abstraction of the dynamics. The measures are illustrated on a climatological example to disentangle pathways of atmospheric flow over Europe.

Figure 1

Consider a realization of dynamical noise ${\eta }^X$ driving subprocess $X$ as a perturbation. Coupling mechanisms along different causal paths (black lines) transform such a perturbation, and the total effect on $Y$ some time later can also depend on how intermediate processes nonlinearly interact with each other as shown in Sec. 5b. The central idea of the momentary information transfer measures presented in this article is to information-theoretically quantify the general effect of such perturbations and isolate it from common drivers in the past such as $Z_2$ but also $Z_1$ and the past of $X$. To also quantify how much intermediate processes such as $(W_1,W_2)$ on causal paths mediate information, it will also be important to exclude common drivers like $Z_3$.

Figure 2

Venn diagrams of (a) mutual information, (b) conditional mutual information, (c) positive interaction information, and (d) negative interaction information. The latter case, where the entropies of $X$ and $Z$ do not “overlap” anymore, demonstrates that the analogy between entropies and sets should not be overinterpreted.

Figure 3

Time series graphs illustrating the path-based measures of information transfer ITX (a) and MITP (b), and process graph (c, the labels denote the lags). Directed links (Def. 7) are marked by arrows, contemporaneous links (Def. 8) by a solid line. There are three causal directed paths connecting $X_{t-3}$ and $Y_t$ (black lines), two of length 2 via $W_{1,t-2}$ and $W_{2,t-1}$ and one of length 3: $X_{t-3}\to W_{1,t-2}\to W_{2,t-1}\to Y_t$. The idea of the measure ITX shown in (a) here is to quantify how much of the information entering the system in $X_{t-3}$, i.e., the dynamical noise ${\eta }^X$, is transferred along causal paths to $Y_t$ by conditioning out the effect of the parents of ${\mathcal{P}}_{X_{t-3}}$ (solid red boxes), its neighbors involving contemporaneous sidepaths to $Y_t$ denoted ${\mathcal{N}}_{X_{t-3}}^{Y_t}$ (dashed red box), and the neighbor's parents $\mathcal{P}\left({\mathcal{N}}_{X_{t-3}}^{Y_t}\right)$ (dotted red boxes). The latter two conditioning sets exclude contemporaneous sidepaths like $X_{t-3}-W_{1,t-3}\to W_{2,t-2}\to Y_{t-1}\to Y_t$. ITX still depends on processes affecting intermediate nodes on causal paths, e.g., process $Z_3$ which drives $W_1$ and $Y$. The idea of MITP shown in (b) now is to go one step further and isolate all causal paths from the remaining process by additionally conditioning on the parents of the intermediate path nodes ${\mathcal{C}}_{X_{t-\tau }\to Y_t}\setminus \left\{X_{t-\tau }\right\}$ (dashed blue boxes) and $Y$ (solid blue boxes). This also allows to isolate mediated effects using momentary interaction information as defined in Sec. 4c.

Figure 4

(a) Time-series graph for model (23) of a causal interaction between three processes at different lags (black dots). The parents are shown in colored boxes, here there are no neighbors. $a,b,c,\alpha$ denote the model coefficients. In (b) the interaction measures are plotted against $a=b$ (strength of the sidepath) and $c$ (strength of direct link) for an autodependency strength $\alpha =0.5$, and in (c) against $c$ and the autodependency strength $\alpha$ for $a=b=0.5$. The color shading only emphasizes the sign and strength, the value can be read off the $z$-axis. All innovation terms $\eta$ have unit variance. Further parameters: ensemble size 30, sample length $T=10,000$, nearest-neighbor estimation parameter $k=1$.

Figure 5

Same as in Fig. 4 but for the nonlinear model (28).

Figure 6

Analysis of daily time series of mean sea level pressure with $T=1268$ days. The algorithm to estimate the parents and neighbors was run as in Ref. [20] using a threshold $I^{*}=0.015\mathrm{nats}$, ${\tau }_{max}=4$ days and the CMI nearest-neighbor parameter $k=100$ (larger $k$ have smaller variance). (a) Anomaly time series (days in winter months November to April only) of the four variables, all units are in hectopascal (hPa) relative to the seasonal mean. (b) Lag functions of MI and multivariate MIT, here a parameter $k=10$ was chosen to reduce the bias. Also contemporaneous MITs as defined in Ref. [48] are shown. All (C)MI values have been rescaled to the (partial) correlation scale via $I\to \sqrt{1-e^{-2I}}\in [0,1]$ [4]. The solid lines denote the fixed threshold $I^{*}=0.015$ (rescaled), which is used to define the time-series graph for the path-analysis. (c) shows the time-series graph with the edge color denoting the rescaled MIT strength. Note the different order of the variables to better visualize the causal paths. Repetitions of links emanating from times further than $t-4$ in the past are omitted. (d) Aggregated visualization as process graph (labels denote the lags and edge and node colors correspond to cross-MIT and auto-MIT, respectively, at the lag with maximum value).