- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Letter
  • Open Access

Fourier-domain transfer entropy spectrum

Yang Tian1,*, Yaoyuan Wang2,†, Ziyang Zhang2,‡, and Pei Sun1,§

  • 1Department of Psychology & Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China
  • 2Data Center Technology Laboratory, Central Research Institute, 2012 Laboratories, Huawei Technologies Company Limited, Beijing 100084, China

  • *tiany20@mails.tsinghua.edu.cn
  • †Corresponding author: wangyaoyuan1@huawei.com
  • ‡Corresponding author: zhangziyang11@huawei.com
  • §Corresponding author: peisun@tsinghua.edu.cn
PDF

Phys. Rev. Research 3, L042040 – Published 15 December, 2021

DOI: https://doi.org/10.1103/PhysRevResearch.3.L042040

Abstract

We propose the Fourier-domain transfer entropy spectrum, a generalization of transfer entropy, as a model-free metric of causality. For arbitrary systems, this approach systematically quantifies the causality among their different system components rather than merely analyzing systems as entireties. The generated spectrum offers a rich-information representation of time-varying latent causal relations, efficiently dealing with nonstationary processes and high-dimensional conditions. We demonstrate its validity in the aspects of parameter dependence, statistical significance tests, and sensibility. An open-source multiplatform implementation of this metric is developed and computationally applied on neuroscience data sets and diffusively coupled logistic oscillators.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (64)

  1. A. Pikovsky, J. Kurths, M. Rosenblum, and J. Kurths, Synchronization: A Universal Concept in Nonlinear Sciences, Cambridge Nonlinear Science Series No. 12 (Cambridge University Press, Cambridge, U.K., 2003).
  2. E. Pereda, R. Q. Quiroga, and J. Bhattacharya, Nonlinear multivariate analysis of neurophysiological signals, Prog. Neurobiol. 77, 1 (2005).
  3. K. Hlaváčková-Schindler, M. Paluš, M. Vejmelka, and J. Bhattacharya, Causality detection based on information-theoretic approaches in time series analysis, Phys. Rep. 441, 1 (2007).
  4. C. W. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica: J. Eco. Soc. 37, 424 (1969).
  5. D. Marinazzo, M. Pellicoro, and S. Stramaglia, Kernel Method for Nonlinear Granger Causality, Phys. Rev. Lett. 100, 144103 (2008).
  6. N. Ancona, D. Marinazzo, and S. Stramaglia, Radial basis function approach to nonlinear Granger causality of time series, Phys. Rev. E 70, 056221 (2004).
  7. M.-C. Ho, Y.-C. Hung, and I.-M. Jiang, Phase synchronization in inhomogeneous globally coupled map lattices, Phys. Lett. A 324, 450 (2004).
  8. A. Arnold, Y. Liu, and N. Abe, Temporal causal modeling with graphical Granger methods, in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, New York, 2007), pp. 66–75.
  9. H. Liu, J. Lafferty, and L. Wasserman, The nonparanormal: Semiparametric estimation of high dimensional undirected graphs, J. Mach. Learn. Res. 10, 2295 (2009).
  10. M. Dhamala, G. Rangarajan, and M. Ding, Estimating Granger Causality from Fourier and Wavelet Transforms of Time Series Data, Phys. Rev. Lett. 100, 018701 (2008).
  11. T. Schreiber, Measuring Information Transfer, Phys. Rev. Lett. 85, 461 (2000).
  12. M. Staniek and K. Lehnertz, Symbolic Transfer Entropy, Phys. Rev. Lett. 100, 158101 (2008).
  13. M. Lobier, F. Siebenhühner, S. Palva, and J. M. Palva, Phase transfer entropy: A novel phase-based measure for directed connectivity in networks coupled by oscillatory interactions, Neuroimage 85, 853 (2014).
  14. M. Lungarella, A. Pitti, and Y. Kuniyoshi, Information transfer at multiple scales, Phys. Rev. E 76, 056117 (2007).
  15. J. Runge, J. Heitzig, V. Petoukhov, and J. Kurths, Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy, Phys. Rev. Lett. 108, 258701 (2012).
  16. P. Wollstadt, M. Martínez-Zarzuela, R. Vicente, F. J. Díaz-Pernas, and M. Wibral, Efficient transfer entropy analysis of non-stationary neural time series, PLoS One 9, e102833 (2014).
  17. M. Porfiri and M. Ruiz Marín, Transfer entropy on symbolic recurrences, Chaos 29, 063123 (2019).
  18. J. F. Restrepo, D. M. Mateos, and G. Schlotthauer, Transfer entropy rate through Lempel-Ziv complexity, Phys. Rev. E 101, 052117 (2020).
  19. J. Zhang, O. Simeone, Z. Cvetkovic, E. Abela, and M. Richardson, ITENE: Intrinsic transfer entropy neural estimator, arXiv:1912.07277.
  20. R. Silini and C. Masoller, Fast and effective pseudo transfer entropy for bivariate data-driven causal inference, Sci. Rep. 11, 8423 (2021).
  21. J. Pearl, Causality: Models, Reasoning and Inference, (Cambridge University Press, Cambridge, U.K., 2000).
  22. C. Diks and V. Panchenko, A note on the Hiemstra-Jones test for Granger non-causality, Stud. Nonlinear Dyn. Econom. 9(2) (2005).
  23. C. Diks and J. DeGoede, A general nonparametric bootstrap test for Granger causality, in Global Analysis of Dynamical Systems, edited by H. W. Broer, B. Krauskopf, and G. Vegter (Institute of Physics Publishing (IOP), London, 2001), pp. 393–405.
  24. R. Vicente, M. Wibral, M. Lindner, and G. Pipa, Transfer entropy—A model-free measure of effective connectivity for the neurosciences, J. Comput. Neurosci. 30, 45 (2011).
  25. J. D. Victor, Binless strategies for estimation of information from neural data, Phys. Rev. E 66, 051903 (2002).
  26. V. A. Vakorin, N. Kovacevic, and A. R. McIntosh, Exploring transient transfer entropy based on a group-wise ICA decomposition of EEG data, Neuroimage 49, 1593 (2010).
  27. R. E. Spinney, M. Prokopenko, and J. T. Lizier, Transfer entropy in continuous time, with applications to jump and neural spiking processes, Phys. Rev. E 95, 032319 (2017).
  28. M. Ursino, G. Ricci, and E. Magosso, Transfer entropy as a measure of brain connectivity: A critical analysis with the help of neural mass models, Front. Comput. Neurosci. 14, 45 (2020).
  29. T. Dimpfl and F. J. Peter, Using transfer entropy to measure information flows between financial markets, Stud. Nonlinear Dyn. Econom. 17, 85 (2013).
  30. A. Papana, C. Kyrtsou, D. Kugiumtzis, and C. Diks, Detecting causality in non-stationary time series using partial symbolic transfer entropy: Evidence in financial data, Comput. Econ. 47, 341 (2016).
  31. F. Toriumi and K. Komura, Investment index construction from information propagation based on transfer entropy, Comput. Econ. 51, 159 (2018).
  32. M. Camacho, A. Romeu, and M. Ruiz-Marin, Symbolic transfer entropy test for causality in longitudinal data, Econ. Model. 94, 649 (2021).
  33. Q. Ji, H. Marfatia, and R. Gupta, Information spillover across international real estate investment trusts: Evidence from an entropy-based network analysis, North Am. J. Econ. Finance 46, 103 (2018).
  34. T. M. Cover, Elements of Information Theory (Wiley, Hoboken, NJ, 1999).
  35. D. A. Smirnov, Spurious causalities with transfer entropy, Phys. Rev. E 87, 042917 (2013).
  36. A. Kraskov, H. Stögbauer, and P. Grassberger, Estimating mutual information, Phys. Rev. E 69, 066138 (2004).
  37. W. A. Gardner, A. Napolitano, and L. Paura, Cyclostationarity: Half a century of research, Signal Process. 86, 639 (2006).
  38. R. J. Marks II, Handbook of Fourier Analysis & its Applications (Oxford University Press, Oxford, U.K., 2009).
  39. P. Brémaud, Mathematical Principles of Signal Processing: Fourier and Wavelet Analysis (Springer, Berlin, 2013).
  40. A. Bruns, Fourier-, Hilbert-and wavelet-based signal analysis: Are they really different approaches? J. Neurosci. Methods 137, 321 (2004).
  41. L. R. Rabiner and B. Gold, Theory and Application of Digital Signal Processing (Prentice-Hall, Englewood Cliffs, NJ, 1975).
  42. O. Rioul and M. Vetterli, Wavelets and signal processing, IEEE Signal Process. Mag. 8, 14 (1991).
  43. N. E. Huang, Hilbert-Huang Transform and its Applications, Vol. 16 (World Scientific, Singapore, 2014).
  44. G. Kaiser and L. H. Hudgins, A Friendly Guide to Wavelets, Vol. 300 (Springer, Berlin, 1994).
  45. G. B. Folland and A. Sitaram, The uncertainty principle: A mathematical survey, J. Fourier Anal. Appl. 3, 207 (1997).
  46. Z. Peng, W. T. Peter, and F. Chu, An improved Hilbert–Huang transform and its application in vibration signal analysis, J. Sound Vib. 286, 187 (2005).
  47. R. T. Ogden, Essential Wavelets for Statistical Applications and Data Analysis (Springer, Berlin, 1997).
  48. C.-L. Liu, A Tutorial of the Wavelet Transform (NTUEE, Taiwan, 2010).
  49. R. R. Coifman, Y. Meyer, and V. Wickerhauser, Wavelet analysis and signal processing, in Wavelets and their Applications (Jones and Barlett, Boston, 1992).
  50. X. Cui, D. M. Bryant, and A. L. Reiss, NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation, Neuroimage 59, 2430 (2012).
  51. X. Cui, D. M. Bryant, and A. L. Reiss, The NIRS data in a built-in directory of Matlab (2012), https://ww2.mathworks.cn/help/wavelet/ug/wavelet-coherence-of-brain-dynamics.html.
  52. C. Bandt and B. Pompe, Permutation Entropy: A Natural Complexity Measure for Time Series, Phys. Rev. Lett. 88, 174102 (2002).
  53. J. Xie, J. Gao, Z. Gao, X. Lv, and R. Wang, Adaptive symbolic transfer entropy and its applications in modeling for complex industrial systems, Chaos 29, 093114 (2019).
  54. C. Torrence and G. P. Compo, A practical guide to wavelet analysis, Bull. Am. Meteorol. Soc. 79, 61 (1998).
  55. S. Panzeri, R. Senatore, M. A. Montemurro, and R. S. Petersen, Correcting for the sampling bias problem in spike train information measures, J. Neurophysiol. 98, 1064 (2007).
  56. R. Andrzejak, A. Ledberg, and G. Deco, Detecting event-related time-dependent directional couplings, New J. Phys. 8, 6 (2006).
  57. T. Wagner, J. Fell, and K. Lehnertz, The detection of transient directional couplings based on phase synchronization, New J. Phys. 12, 053031 (2010).
  58. M. Martini, T. A. Kranz, T. Wagner, and K. Lehnertz, Inferring directional interactions from transient signals with symbolic transfer entropy, Phys. Rev. E 83, 011919 (2011).
  59. B. Phipson and G. K. Smyth, Permutation p-values should never be zero: Calculating exact p-values when permutations are randomly drawn, Stat. Appl. Genet. Mol. Biol. 9(1) (2010).
  60. E. Maris and R. Oostenveld, Nonparametric statistical testing of EEG- and MEG-data, J. Neurosci. Methods 164, 177 (2007).
  61. M. Lindner, R. Vicente, V. Priesemann, and M. Wibral, TRENTOOL: A Matlab open source toolbox to analyse information flow in time series data with transfer entropy, BMC Neurosci. 12, 119 (2011).
  62. A. L. Lloyd, The coupled logistic map: A simple model for the effects of spatial heterogeneity on population dynamics, J. Theor. Biol. 173, 217 (1995).
  63. A. Valencio and M. d. S. Baptista, Coupled logistic maps: Functions for generating the time-series from networks of coupled logistic systems (2018), open source codes for MATLAB. available at https://github.com/artvalencio/coupled-logistic- maps.
  64. Y. Tian, Y. Wang, Z. Zhang, and P. Sun, Transfer entropy spectrum in the Fourier domain (2021), open source codes available at https://github.com/doloMing/Transfer-entropy-spectrum-in-the-Fourier-domain.

Outline

Information

Phys. Rev. Research 3, L042040– Published 15 December, 2021
  • Received 12 October 2021
  • Accepted 30 November 2021
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation