Characterizing time series via complexity-entropy curves

The search for patterns in time series is a very common task when dealing with complex systems. This is usually accomplished by employing a complexity measure such as entropies and fractal dimensions. However, such measures usually only capture a single aspect of the system dynamics. Here, we propose a family of complexity measures for time series based on a generalization of the complexity-entropy causality plane. By replacing the Shannon entropy by a monoparametric entropy (Tsallis $q$ entropy) and after considering the proper generalization of the statistical complexity ($q$ complexity), we build up a parametric curve (the $q$-complexity-entropy curve) that is used for characterizing and classifying time series. Based on simple exact results and numerical simulations of stochastic processes, we show that these curves can distinguish among different long-range, short-range, and oscillating correlated behaviors. Also, we verify that simulated chaotic and stochastic time series can be distinguished based on whether these curves are open or closed. We further test this technique in experimental scenarios related to chaotic laser intensity, stock price, sunspot, and geomagnetic dynamics, confirming its usefulness. Finally, we prove that these curves enhance the automatic classification of time series with long-range correlations and interbeat intervals of healthy subjects and patients with heart disease.

Figure 1

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for the fractional Brownian motion. Panels (a) and (b) show the $q$-complexity-entropy curves with $d=3$ and several values of the Hurst exponent $h$ [$h$ in $(0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9)$, as indicated in the plots]. The values of $q$ are increasing (from $q=0^{+}$ to $q=1000$ in size steps of ${10}^{-4}$) in the clockwise direction. Panels (c) and (d) show the same, but with embedding dimension $d=4$. All colored curves represent the average value of $H_q$ and $C_q$ over 100 realizations of a fractional Brownian walker with $2^{17}$ steps. The shaded areas indicate the 95% confidence intervals estimated via bootstrapping (over 100 independent realizations). The dashed lines are the exact results calculated by using the probabilities of Bandt and Shiha [38] (see also Appendix pp2). It is worth noting that all curves form loops in this representation space, starting at $(1,0)$ for $q=0^{+}$ and ending at $(1,0)$ for $q\to \infty$.

Figure 2

Comparison between the extreme values $q_H^{*}$ and $q_C^{*}$ obtained from the simulations and the exact results for the fractional Brownian motion. Panel (a) shows the values of $q=q_H^{*}$ for which $H_q$ reaches a minimum as a function of the Hurst exponent $h$ and $d=3$. Each dot corresponds to the average value of $q_H^{*}$ obtained from 100 realizations of a fractional Brownian walker with $2^{17}$ steps. Error bars stand for 95% confidence intervals estimated via bootstrapping (over 100 independent realizations). Panel (b) shows the values of $q=q_C^{*}$ for which $C_q$ reaches a maximum as a function of the Hurst exponent $h$ and $d=3$. Again, the dots are the average value calculated from 100 independent realizations of a fractional Brownian walker with $2^{17}$ steps, and the error bars are 95% confidence intervals. In both plots, the continuous lines are the exact results. Panels (c) and (d) are the analogous of (a) and (b) when considering the embedding dimension $d=4$. In all cases, we note an excellent agreement between the simulations and the exact results.

Figure 3

Predicting the Hurst exponent based on the values of $H_q$ and $C_q$ via the nearest neighbors algorithm. (a) Confusion matrices obtained from the nearest neighbors algorithm when considering the values of $H_q$ and $C_q$ for $q=1$ (usual case, left panel) and the optimized values of $H_{q_H^{*}}$ and $C_{q_C^{*}}$ (optimized case, right panel) for time series of length $n=2^{10}$. The rows of these matrices represent the actual Hurst exponents (the value employed in the simulation) and the columns represent the values predicted by the machine learning algorithm. Values of the Hurst exponents vary from 0.03 to 0.97 in steps of size 0.02. The color code indicates the fraction of occurrences for each combination of actual and predicted Hurst exponent. In (b) we show the same analysis for longer times series ($n=2^{12}$). We note that in both classification scenarios the predicted values are always very close to the actual values (that is, the confusion matrices have “diagonal stripes”). However, we further notice that the “diagonal stripe” is narrower in the optimized case (especially for persistent processes). (c) Overall accuracy (fraction of correctly classified Hurst exponents) for the two classification scenarios (error bars are the standard errors) for different time series length. We notice that the use of $H_q$ and $C_q$ with $q=q_H^{*}$ and $q=q_C^{*}$ (optimized case) provides a greater overall accuracy when compared with the case $q=1$, regardless of the length $n$.

Figure 4

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for the harmonic noise: changes with the frequency parameter $\omega$. Panel (a) shows the $q$-complexity-entropy curves with embedding dimension $d=3, \mathrm{\Gamma }=0.05, \varepsilon =1$, and some values of the parameter $\omega$ (shown in the plot). The values of $q$ are increasing (from $q=0^{+}$ to 1000 in size steps of ${10}^{-4}$) in the clockwise direction. The two insets highlight the regions of the causality plane where the entropy reaches a minimum ($q=q_H^{*}$, indicated by cross markers) and the complexity passes to its maximum value ($q=q_C^{*}$, indicated by asterisk markers). The points ($H_q, C_q$) for $q=1$ are indicated by black dots. Panel (b) shows the dependence of $H_q$ on $q$ for several values of $\omega$ (indicated by the color code) and the inset highlights the region where the minimums occur ($q=q_H^{*}$, indicated by black dots). Panel (c) shows the dependence of $C_q$ on $q$ for several values of $\omega$ [the same of panel (b)] and the inset highlights the region where the maximums occur ($q=q_C^{*}$, indicated by black dots). Panels (d) and (e) show the dependence of the extreme values of $q$ ($q_H^{*}$ and $q_C^{*}$) on the frequency parameter $\omega$ (the markers are the average values over 100 realizations of the harmonic noise with maximum integration time of 1320 and step size of ${10}^{-3}$, and the error bars stand for 95% bootstrap confidence intervals). We note that $q_H^{*}$ monotonically increases with $\omega$ in a relationship that is approximated by an exponential approach to the value $q_H^{*}=0.867$ (that is, $q_H^{*}=0.867-0.279e^{-0.018\omega }$, as indicated by the continuous line). We further notice that $q_C^{*}$ decreases with $\omega$ and that around $\omega =16.5$ there is a discontinuous behavior. The continuous lines in this last plot are linear approximations to the behavior of $q_C^{*}$.

Figure 5

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for the harmonic noise: changes with the damping coefficient $\mathrm{\Gamma }$. Panel (a) shows $q$-complexity-entropy curves with embedding dimension $d=3, \omega ={10}^{-4}, \varepsilon =1$, and for some values of the parameter $\mathrm{\Gamma }$ (shown in the plot). The values of $q$ are increasing (from $q=0^{+}$ to 1000 in size steps of ${10}^{-4}$) in the clockwise direction. The black dots indicate the points ($H_q, C_q$) for $q=1$, cross markers for $q=q_H^{*}$, and asterisk markers for $q=q_C^{*}$. Panel (b) shows the dependence of $H_q$ on $q$ for several values of $\mathrm{\Gamma }$ (indicated by the color code), where the black dots indicate the values of $q=q_H^{*}$ that minimize $H_q$. Panel (c) shows the dependence of $C_q$ on $q$ for several values of $\mathrm{\Gamma }$ [the same of panel (b)], where the black dots indicate the values of $q=q_C^{*}$ that maximize $C_q$. Panels (d) and (e) show the dependence of the extreme values of $q$ ($q_H^{*}$ and $q_C^{*}$) on the damping parameter $\mathrm{\Gamma }$ (the markers are the average values over 100 realizations of the harmonic noise with maximum integration time of 1320 and step size of ${10}^{-3}$, and the error bars stand for 95% bootstrap confidence intervals). We note that $q_H^{*}$ logarithmically increases with $\mathrm{\Gamma }$ up to $\mathrm{\Gamma }\approx 300$ [$q_H^{*}=0.73+0.07*ln\left(\mathrm{\Gamma }\right)$, as indicated by the continuous line], where it saturates around $q_H^{*}\approx 1.12$ (continuous line). We further notice that $q_C^{*}$ logarithmically decreases with $\mathrm{\Gamma }$ up to $\mathrm{\Gamma }\approx 1000$ [$q_C^{*}=5.70-0.47*ln\left(\mathrm{\Gamma }\right)$, as indicated by the continuous line], where it saturates around $q_C^{*}\approx 2.53$ (continuous line).

Figure 6

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for chaotic maps at fully developed chaos and stochastic processes. Each plot shows the $q$-complexity-entropy curve for a chaotic map (first two rows) or a stochastic process (last two rows) for embedding dimensions $d=3,4,5,6$ (the different colors). The first two rows show the results for the chaotic maps: Burgers, cubic, Gingerbreadman, Henon, logistic, Ricker, sine, and Tinkerbell at fully developed chaos (see Appendix pp3 for more detail). The third row refers to the fractional Brownian motion (Hurst exponent $h$ is indicated in the plots) and the last row refers to the harmonic noise (parameters $\mathrm{\Gamma }$ and $\omega$ are indicated in the plots). In each panel, the star markers indicate the points $(H_q,C_q)$ for $q=0^{+}$, while the open circles are the same for $q\to \infty$. We note that stochastic processes are mostly characterized by loops for all embedding dimensions, whereas chaotic maps usually form open curves in the causality plane. We further note that, differently from stochastic processes, $H_q$ of chaotic maps does not exhibit a minimum value for all embedding dimensions.

Figure 7

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for the logistic map. Panel (a) shows a comparison between the values $H_q$ and $C_q$ (as well as their dependence on $q$) obtained from the simulations and the exact results for the logistic map with $a=4$ and $d=3$. Notice that a practically perfect agreement is found. Panel (b) shows the $q$-complexity-entropy curve and the dependence of $H_q$ and $C_q$ on $q$ for $d=4$ and four values of the parameter $a$: $a=3.05$ (oscillating behavior between two values), $a=3.50$ (oscillating behavior among four values), $a=3.593$ (chaotic behavior), and $a=4$ (fully developed chaos). We note that the complete differentiation among these regimes of the logistic map is only possible when considering different values of $q$. In particular, we observe that the points $(H_q,C_q)$ for $q=1$ (indicated by the black dots) are in about the same location for $a=3.50$ and 3.593.

Figure 8

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for empirical time series. Panel (a) shows the $q$-complexity-entropy curve for the chaotic intensity pulsations of a single-mode far-infrared NH3 laser. Panel (b) shows the curves for crude oil prices (daily closing spot price of the West Texas Intermediate from January 2, 1986 to July 10, 2012), and panel (c) for the monthly smoothed sunspots index (from 1749 to 2016). In all panels, the different colors refer to the embedding dimensions ($d=3,4,5$), the star markers indicate the points $(H_q,C_q)$ for $q=0^{+}$, while the open circles are the same for $q\to \infty$. Also, the gray dots indicate the points ($H_q, C_q$) for $q=1$, cross markers for $q=q_H^{*}$, and asterisk markers for $q=q_C^{*}$. We note that causality plane for the laser intensity is similar to those reported for chaotic maps, while the crude oil prices and sunspot index have a behavior similar to those reported for noisy time series (see Fig. 6). (d) Extreme values $q_H^{*}$ and $q_C^{*}$ obtained for each system and $d=3,4,5$ (different bar colors).

Figure 9

Distinguishing between the interbeat intervals of healthy subjects and patients with congestive heart failure based on the values of $H_q$ and $C_q$ via the nearest neighbors algorithm. Panel (a) shows the causality plane (for the embedding dimension $d=3$) evaluated from heart rate time series of 15 patients (age $=69.7\pm 6.4$) with severe congestive heart failure (NYHA class III, gray curve) and from the time series of 46 healthy subjects (age $=65.9\pm 4.0$, green curve). The continuous lines are the average values of $H_q$ and $C_q$ over all subjects in each group and the shaded areas are the standard error of the mean values. Panel (b) shows the accuracy of the nearest neighbors algorithm (fraction of correctly classified subjects) when employing the optimized values of $H_q$ and $C_q$ ($q=q_H^{*}$ and $q=q_C^{*}$, blue bar) and when using the values of $H_q$ and $C_q$ for $q=1$ (usual case, red bar). The error bars are 95% confidence intervals calculated via cross validation. We notice that the optimized values of $H_q$ and $C_q$ provide a greater accuracy when compared with the usual case ($\approx 80\%$ against $\approx 76\%$).

Figure 10

Dependence of the entropy $H_q$ and complexity $C_q$ on the parameter $q$ for the Earth's magnetic activity: changes during a geomagnetic storm. Panel (a) shows the hourly time series of the disturbance storm time index (DST, a measure of the Earth magnetic activity) from January 1, 1989 to May 24, 1989 (144 days). Within this period, a severe geomagnetic storm struck Earth on March 13, 1898, when the DST dropped to about $-600\mathrm{nT}$. The time series is segmented in 8 periods (indicated by different colors) of 18 days and the period containing the geomagnetic storm is plotted in black. Panel (b) shows the $q$-complexity-entropy curves evaluated for each 18-day period for the embedding dimension $d=3$. The gray dots indicate the points ($H_q, C_q$) for $q=1$, cross markers for $q=q_H^{*}$, and asterisk markers for $q=q_C^{*}$. We notice that during the geomagnetic storm the $q$-complexity-entropy curve has the smallest value for the entropy $H_q$ and largest value for the complexity $C_q$ (that is, a broader loop). It is also worth mentioning that the period just after the storm is characterized by the shortest loop. (c) Extreme values $q_H^{*}$ and $q_C^{*}$ obtained for time series segment. (d) The $q$-complexity entropy curves evaluated for shuffled versions of each 18-day period. The continuous lines are the average values of curves over 100 realizations and the shaded areas indicate 95% bootstrap confidence intervals. The color code employed in each plot is the same used for the DST time series.