- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Feature-based information-theoretic approach for detecting interpretable, long-timescale pairwise interactions from time series

Aria Nguyen1,2, Oscar McMullin1, Joseph T. Lizier2,3, and Ben D. Fulcher1,2,*

  • *Contact author: ben.fulcher@sydney.edu.au
PDF

Phys. Rev. Research 7, 043283 – Published 10 December, 2025

DOI: https://doi.org/10.1103/qnx2-yp4c

Abstract

Quantifying relationships between components of a complex system is critical to understanding the rich network of interactions that characterize the behavior of the system. Traditional methods for detecting pairwise dependence of time series, such as Pearson correlation, Granger causality, and mutual information, are computed directly in the space of measured time-series values. However, for systems in which interactions are mediated by statistical properties of the time series (time-series features) over longer timescales, this approach can fail to capture the underlying dependence from limited and noisy time-series data and can be challenging to interpret. Addressing these issues, here we introduce an information-theoretic method for detecting dependence between time series mediated by time-series features that provides interpretable insights into the nature of the interactions. Our method extracts a candidate set of time-series features from sliding windows of the source time series and assesses their role in mediating a relationship to values of the target process. Across simulations of three different generative processes, we demonstrate that our feature-based approach can outperform a traditional inference approach based on raw time-series values, especially in challenging scenarios characterized by short time-series lengths, high noise levels, and long interaction timescales. Our work introduces a tool for inferring and interpreting feature-mediated interactions from time-series data, contributing to the quantitative research of complex systems more broadly, with potential applications in various domains including but not limited to neuroscience, finance, climate science, and engineering.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (64)

  1. L. Peel, T. P. Peixoto, and M. De Domenico, Statistical inference links data and theory in network science, Nat. Commun. 13, 6794 (2022).
  2. J. Runge et al., Inferring causation from time series in Earth system sciences, Nat. Commun. 10, 2553 (2019).
  3. T. Hoffmann, L. Peel, R. Lambiotte, and N. S. Jones, Community detection in networks without observing edges, Sci. Adv. 6, eaav1478 (2020).
  4. S. Lefèvre, D. Vasquez, and C. Laugier, A survey on motion prediction and risk assessment for intelligent vehicles, ROBOMECH J. 1, 1 (2014).
  5. X. Li et al., Quasi-periodic oscillations of the x-ray burst from the magnetar SGR J1935–2154 and associated with the fast radio burst FRB 200428, Astrophys. J. 931, 56 (2022).
  6. D. Freedman, R. Pisani, and R. Purves, in Statistics (International Student Edition), edited by R. Pisani, R. Purves, 4th ed. (WW Norton & Company, New York, 2007).
  7. C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, in Essays in Econometrics: Collected Papers of Clive W. J. Granger (Harvard University Press, USA, 2001), pp. 31–47.
  8. O. M. Cliff, A. G. Bryant, J. T. Lizier, N. Tsuchiya, and B. D. Fulcher, Unifying pairwise interactions in complex dynamics, Nat. Comput. Sci. 3, 883 (2023).
  9. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms (Cambridge University Press, Cambridge, 2003).
  10. T. Schreiber, Measuring information transfer, Phys. Rev. Lett. 85, 461 (2000).
  11. T. R. J. Bossomaier, L. C. Barnett, M. Harré, and J. T. Lizier, An Introduction to Transfer Entropy: Information Flow in Complex Systems (Springer, Cham, 2016).
  12. M. Materassi, G. Consolini, N. Smith, and R. De Marco, Information theory analysis of cascading process in a synthetic model of fluid turbulence, Entropy 16, 1272 (2014).
  13. M. Ursino, G. Ricci, and E. Magosso, Transfer entropy as a measure of brain connectivity: A critical analysis with the help of neural mass models, Front. Comput. Neurosci. 14, 45 (2020).
  14. T. Dimpfl and F. J. Peter, Using transfer entropy to measure information flows between financial markets, Stud. Nonlinear Dyn. Econom. 17, 85 (2013).
  15. J. Li, C. Liang, X. Zhu, X. Sun, and D. Wu, Risk contagion in chinese banking industry: A transfer entropy-based analysis, Entropy 15, 5549 (2013).
  16. Y. Li and D. E. Giles, Modelling volatility spillover effects between developed stock markets and Asian emerging stock markets, Int. J. Finance Econ. 20, 155 (2015).
  17. M. Bonnefond, S. Kastner, and O. Jensen, Communication between brain areas based on nested oscillations, eNeuro 4, 0153 (2017).
  18. M. Markicevic, B. D. Fulcher, C. Lewis, F. Helmchen, M. Rudin, V. Zerbi, and N. Wenderoth, Cortical excitation: Inhibition imbalance causes abnormal brain network dynamics as observed in neurodevelopmental disorders, Cereb. Cortex 30, 4922 (2020).
  19. S. Trakoshis, P. Martínez-Cañada, F. Rocchi, C. Canella, W. You, B. Chakrabarti, A. N. V. Ruigrok, E. T. Bullmore, J. Suckling, M. Markicevic, V. Zerbi, MRC AIMS Consortium, S. Baron-Cohen, A. Gozzi, M.-C. Lai, S. Panzeri, and M. V. Lombardo, Intrinsic excitation-inhibition imbalance affects medial prefrontal cortex differently in autistic men versus women, eLife 9, e55684 (2020).
  20. L. Albuquerque, A. R. S. Valente, A. Teixeira, D. Figueiredo, P. Sa-Couto, and C. Oliveira, Association between acoustic speech features and non-severe levels of anxiety and depression symptoms across lifespan, PLoS One 16, 1 (2021).
  21. F. Takens, Detecting strange attractors in turbulence, in Dynamical Systems and Turbulence, Warwick 1980, edited by D. Rand and L.-S. Young, Lecture Notes in Mathematics (Springer, Berlin, 1981), Chap. 21, Vol. 898, pp. 366–381.
  22. J. T. Lizier, The Local Information Dynamics of Distributed Computation in Complex Systems, Springer Theses, 2013th ed. (Springer, Berlin, Germany, 2012).
  23. T. Edinburgh, S. J. Eglen, and A. Ercole, Causality indices for bivariate time series data: A comparative review of performance, Chaos 31, 083111 (2021).
  24. R. Bellman, Dynamic programming, Science 153, 34 (1966).
  25. D. Donoho, High-dimensional data analysis: The curses and blessings of dimensionality, in AMS Math Challenges Lecture (2000), pp. 1–32.
  26. M. Staniek and K. Lehnertz, Symbolic transfer entropy, Phys. Rev. Lett. 100, 158101 (2008).
  27. M. Zanin, L. Zunino, O. A. Rosso, and D. Papo, Permutation entropy and its main biomedical and econophysics applications: A review, Entropy 14, 1553 (2012).
  28. D. P. Shorten, R. E. Spinney, and J. T. Lizier, Estimating transfer entropy in continuous time between neural spike trains or other event-based data, PLoS Comput. Biol. 17, e1008054 (2021).
  29. B. D. Fulcher, Feature-based time-series analysis, in Feature Engineering for Machine Learning and Data Analytics (CRC Press, Boca Raton, Florida, USA, 2018).
  30. T. Henderson and B. D. Fulcher, An empirical evaluation of time-series feature sets, in Proceedings of 2021 International Conference on Data Mining Workshops (ICDMW) (2021), pp. 1032–1038.
  31. B. D. Fulcher, M. A. Little, and N. S. Jones, Highly comparative time-series analysis: The empirical structure of time series and their methods, J. R. Soc. Interface 10, 20130048 (2013).
  32. B. D. Fulcher and N. S. Jones, hctsa: A computational framework for automated time-series phenotyping using massive feature extraction, Cell Syst. 5, 527 (2017).
  33. C. H. Lubba, S. Sethi, P. Knaute, S. Schultz, B. Fulcher, and N. Jones, Catch22: CAnonical Time-series CHaracteristics: Selected through highly comparative time-series analysis, Data Min. Knowl. Discovery 33, 1821 (2019).
  34. M. Christ, N. Braun, J. Neuffer, and A. W. Kempa-Liehr, Time series FeatuRe extraction on basis of scalable hypothesis tests (tsfresh—a Python package), Neurocomputing 307, 72 (2018).
  35. M. O’Hara-Wild, R. Hyndman, and E. Wang, feasts: Feature extraction and statistics for time series, R package version 0.4.2, 2021, https://github.com/tidyverts/feasts.
  36. T. Cover and J. Thomas, Information theory and statistics, in Elements of Information Theory (John Wiley & Sons, Ltd, 2005), Chap. 11, pp. 347–408.
  37. J. T. Lizier, JIDT: An information-theoretic toolkit for studying the dynamics of complex systems, Front. Rob. AI 1, (2014).
  38. P. F. Verdes, Assessing causality from multivariate time series, Phys. Rev. E 72, 026222 (2005).
  39. R. Vicente, M. Wibral, M. Lindner, and G. Pipa, Transfer entropy—A model-free measure of effective connectivity for the neurosciences, J. Comput. Neurosci. 30, 45 (2011).
  40. S. Holm, A simple sequentially rejective multiple test procedure, Scand. J. Stat. 6, 65 (1979).
  41. A. Kraskov, H. Stögbauer, and P. Grassberger, Estimating mutual information, Phys. Rev. E 69, 066138 (2004).
  42. O. M. Cliff, L. Novelli, B. D. Fulcher, J. M. Shine, and J. T. Lizier, Assessing the significance of directed and multivariate measures of linear dependence between time series, Phys. Rev. Res. 3, 013145 (2021).
  43. B. Goswami, A brief introduction to nonlinear time series analysis and recurrence plots, Vibration 2, 332 (2019).
  44. T. Schreiber and A. Schmitz, Surrogate time series, Physica D 142, 346 (2000).
  45. R. Vicente and M. Wibral, Efficient estimation of information transfer, in Directed Information Measures in Neuroscience, edited by M. Wibral, R. Vicente, and J. T. Lizier (Springer, Berlin, 2014), pp. 37–58.
  46. M. Wibral, R. Vicente, and M. Lindner, Transfer entropy in neuroscience, in Directed Information Measures in Neuroscience, edited by M. Wibral, R. Vicente, and J. T. Lizier (Springer, Berlin, 2014), pp. 3–36.
  47. S. J. Leybourne, B. P. M. McCabe, and A. R. Tremayne, Can economic time series be differenced to stationarity? J. Bus. Econ. Stat. 14, 435 (1996).
  48. M. Cassidy and W. Penny, Bayesian nonstationary autoregressive models for biomedical signal analysis, IEEE Trans. Biomed. Eng. 49, 1142 (2002).
  49. G. E. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control (Holden-Day, San Francisco, California, USA, 1976), Chap. 3.
  50. B. Venables and B. Ripley, Modern Applied Statistics with S (Springer, New York, 2002).
  51. D. Feldman, The spike-timing dependence of plasticity, Neuron 75, 556 (2012).
  52. W. M. Usrey, J.-M. Alonso, and R. C. Reid, Synaptic interactions between thalamic inputs to simple cells in cat visual cortex, J. Neurosci. 20, 5461 (2000).
  53. W. Gerstner, A. K. Kreiter, H. Markram, and A. V. M. Herz, Neural codes: Firing rates and beyond, Proc. Natl. Acad. Sci. USA 94, 12740 (1997).
  54. T. Schreiber, Detecting and analyzing nonstationarity in a time series using nonlinear cross predictions, Phys. Rev. Lett. 78, 843 (1997).
  55. L. A. Aguirre and C. Letellier, Nonstationarity signatures in the dynamics of global nonlinear models, Chaos 22, 033136 (2012).
  56. V. Griffith and C. Koch, Quantifying synergistic mutual information, in Guided Self-Organization: Inception (Springer, Berlin, 2014), pp. 159–190.
  57. N. Timme, W. Alford, B. Flecker, and J. M. Beggs, Synergy, redundancy, and multivariate information measures: An experimentalist's perspective, J. Comput. Neurosci. 36, 119 (2014).
  58. R. Quax, O. Har-Shemesh, and P. M. A. Sloot, Quantifying synergistic information using intermediate stochastic variables, Entropy 19, 85 (2017).
  59. C. J. Rozell and D. H. Johnson, Analyzing the robustness of redundant population codes in sensory and feature extraction systems, Neurocomputing 69, 1215 (2006).
  60. C. Rozell and D. Johnson, Analysis of noise reduction in redundant expansions under distributed processing requirements, in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’05) (2005), Vol. 4, pp. iv/185–iv/188.
  61. M. Wibral, N. Pampu, V. Priesemann, F. Siebenhühner, H. Seiwert, M. Lindner, J. T. Lizier, and R. Vicente, Measuring Information-Transfer Delays, PLoS One 8, e55809 (2013).
  62. N. Tishby, F. C. Pereira, and W. Bialek, The information bottleneck method, arXiv:physics/0004057.
  63. A. Nguyen, Feature based dependence detection - Simulation results, figshare, Dataset (2025), https://doi.org/10.6084/m9.figshare.30070102.v3.
  64. A. Nguyen, DynamicsAndNeuralSystems/feature-mediated-dependency-detection: v0.1.0 (v0.1), Zenodo (2025), doi:10.5281/zenodo.17188608.

Outline

Information

Phys. Rev. Research 7, 043283– Published 10 December, 2025
  • Received 1 May 2024
  • Accepted 14 October 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation