Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Statistical resampling of time series by permutation

San To Chan* and Eliot Fried†

  • *Contact author: san.chan@oist.jp
  • †Contact author: eliot.fried@oist.jp

Phys. Rev. Research 8, 043169 – Published 17 November, 2026

DOI: https://doi.org/10.1103/g6fz-4fsh

Abstract

We propose a statistical resampling scheme that generates an ensemble of permuted time series by repeatedly swapping randomly selected pairs of adjacent data points. In the infinite-ensemble limit, we derive the exact ensemble-average operator, which describes how the ensemble average evolves with the number of swaps and satisfies a discrete diffusion equation. Under a periodic idealization, the corresponding transfer function reduces asymptotically to the Gaussian response in the low-frequency limit when the number of swaps is on the order of the data length. The ensemble-average operator also affords a dimensionless relation involving the residue between the original and ensemble-averaged time series, the magnitude of the first difference, the number of swaps, and the data length. When the time series is dominated by white noise, this relation reduces to a similarity function that depends solely on the normalized number of swaps. This result motivates a procedure based on the Kolmogorov-Smirnov test for selecting the extent of permutation according to approximate preservation of the marginal first-difference distribution. Under the periodic idealization, the characteristic function of the random displacement of an individual data value is identical to the transfer function of the ensemble-average operator, thereby providing a destructive interference interpretation of the operator’s low-pass behavior. Finally, the individual permuted time series are demonstrated as surrogate inputs to downstream analyses in a proof-of-concept application using empirical mode decomposition.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (46)

  1. J. Klapper and C. Harris, On the response and approximation of Gaussian filters, IRE Trans. Audio AU-7, 80 (1959).
  2. S. Ohkawa, M. Yoshizawa, and K. Husimi, Direct synthesis of the Gaussian filter for nuclear pulse amplifiers, Nucl. Instrum. Methods 138, 85 (1976).
  3. M. Basu, Gaussian-based edge-detection methods—A survey, IEEE Trans. Syst. Man Cybern. C 32, 252 (2002).
  4. F. Dablander and T. M. Bury, Deep learning for tipping points: Preprocessing matters, Proc. Natl. Acad. Sci. USA 119, e2207720119 (2022).
  5. J. Nako, G. Tsirimokou, C. Psychalinos, A. S. Elwakil, B. J. Maundy, and P. Bertsias, Reduced complexity approximation and design of Gaussian impulse response filters and wavelets, IEEE Access 12, 160064 (2024).
  6. See Supplemental Material at http://link.aps.org/supplemental/10.1103/g6fz-4fsh for matlab code implementing the synthetic ensemble generation workflow.
  7. A. Kolmogorov, Sulla determinazione empirica di una legge di distribuzione, G. Ist. Ital. Attuari 4, 83 (1933).
  8. N. Smirnov, Table for estimating the goodness of fit of empirical distributions, Ann. Math. Stat. 19, 279 (1948).
  9. F. J. Massey Jr., The Kolmogorov-Smirnov test for goodness of fit, J. Am. Stat. Assoc. 46, 68 (1951).
  10. G. Marsaglia, W. W. Tsang, and J. Wang, Evaluating Kolmogorov’s distribution, J. Stat. Software 8, 1 (2003).
  11. G. B. Arfken, H. J. Weber, and F. E. Harris, Mathematical Methods for Physicists: A Comprehensive Guide (Academic Press, Massachusetts, USA, 2011).
  12. A. Savitzky and M. J. E. Golay, Smoothing and differentiation of data by simplified least squares procedures, Anal. Chem. 36, 1627 (1964).
  13. R. W. Schafer, What is a Savitzky–Golay filter? IEEE Signal Process. Mag. 28, 111 (2011).
  14. J. D. Hart, Kernel regression estimation with time series errors, J. R. Stat. Soc. B: Stat. Methodol. 53, 173 (1991).
  15. W. Härdle and P. Vieu, Kernel regression smoothing of time series, J. Time Ser. Anal. 13, 209 (1992).
  16. I. J. Schoenberg, Contributions to the problem of approximation of equidistant data by analytic functions. Part A. On the problem of smoothing or graduation. A first class of analytic approximation formulae, Q. Appl. Math. 4, 45 (1946).
  17. C. H. Reinsch, Smoothing by spline functions, Numer. Math. 10, 177 (1967).
  18. P. Craven and G. Wahba, Smoothing noisy data with spline functions, Numer. Math. 31, 377 (1978).
  19. R. R. Picard and R. D. Cook, Cross-validation of regression models, J. Am. Stat. Assoc. 79, 575 (1984).
  20. J. Shao, Linear model selection by cross-validation, J. Am. Stat. Assoc. 88, 486 (1993).
  21. L. A. Yates, Z. Aandahl, S. A. Richards, and B. W. Brook, Cross validation for model selection: A review with examples from ecology, Ecol. Monogr. 93, e1557 (2023).
  22. S. Bates, T. Hastie, and R. Tibshirani, Cross-validation: What does it estimate and how well does it do it? J. Am. Stat. Assoc. 119, 1434 (2024).
  23. A. E. Hoerl and R. W. Kennard, Ridge regression: Biased estimation for nonorthogonal problems, Technometrics 12, 55 (1970).
  24. R. Tibshirani, Regression shrinkage and selection via the lasso, J. R. Stat. Soc. B: Stat. Methodol. 58, 267 (1996).
  25. P. H. C. Eilers and B. D. Marx, Flexible smoothing with B-splines and penalties, Stat. Sci. 11, 89 (1996).
  26. P. H. C. Eilers and B. D. Marx, Splines, knots, and penalties, WIREs Interdiscip. Rev. Comput. Stat. 2, 637 (2010).
  27. T. Lindeberg, Scale-space for discrete signals, IEEE Trans. Pattern Anal. Mach. Intell. 12, 234 (1990).
  28. H. Lacoin, Mixing time and cutoff for the adjacent transposition shuffle and the simple exclusion, Ann. Probab. 44, 1426 (2016).
  29. K. Kawasaki, Diffusion constants near the critical point for time-dependent Ising models. I, Phys. Rev. 145, 224 (1966).
  30. K. Kawasaki, Diffusion constants near the critical point for time-dependent Ising models. II, Phys. Rev. 148, 375 (1966).
  31. C. Bandt and B. Pompe, Permutation entropy: A natural complexity measure for time series, Phys. Rev. Lett. 88, 174102 (2002).
  32. T. Schreiber, Constrained randomization of time series data, Phys. Rev. Lett. 80, 2105 (1998).
  33. T. Schreiber and A. Schmitz, Surrogate time series, Physica D 142, 346 (2000).
  34. N. E. Huang, Z. Shen, S. R. Long, M. C. Wu, H. H. Shih, Q. Zheng, N. C. Yen, C. C. Tung, and H. H. Liu, The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis, Proc. R. Soc. A 454, 903 (1998).
  35. A. Zeiler, R. Faltermeier, I. R. Keck, A. M. Tomé, C. G. Puntonet, and E. W. Lang, Empirical mode decomposition—An introduction, in The 2010 International Joint Conference on Neural Networks (IJCNN) (IEEE, Barcelona, 2010), pp. 1–8.
  36. MathWorks, EMD—Empirical mode decomposition, MATLAB Signal Processing Toolbox, https://www.mathworks.com/help/signal/ref/emd.html.
  37. L. N. Trefethen and D. Bau, III, Numerical Linear Algebra (Society for Industrial and Applied Mathematics, Philadelphia, PA, 1997).
  38. Z. Wu and N. E. Huang, Ensemble empirical mode decomposition: A noise-assisted data analysis method, Adv. Adapt. Data Anal. 01, 1 (2009).
  39. M. E. Torres, M. A. Colominas, G. Schlotthauer, and P. Flandrin, A complete ensemble empirical mode decomposition with adaptive noise, in 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, Prague, 2011), pp. 4144–4147.
  40. E. J. Kostelich and T. Schreiber, Noise reduction in chaotic time-series data: A survey of common methods, Phys. Rev. E 48, 1752 (1993).
  41. V. Plerou, P. Gopikrishnan, B. Rosenow, L. A. N. Amaral, and H. E. Stanley, Econophysics: Financial time series from a statistical physics point of view, Physica A 279, 443 (2000).
  42. J. Aljadeff, B. J. Lansdell, A. L. Fairhall, and D. Kleinfeld, Analysis of neuronal spike trains, deconstructed, Neuron 91, 221 (2016).
  43. H. Y. D. Sigaki, M. Perc, and H. V. Ribeiro, History of art paintings through the lens of entropy and complexity, Proc. Natl. Acad. Sci. USA 115, E8585 (2018).
  44. B. Lee, M. K. Seo, D. Kim, I. Shin, M. Schich, H. Jeong, and S. K. Han, Dissecting landscape art history with information theory, Proc. Natl. Acad. Sci. USA 117, 26580 (2020).
  45. A. A. Bagrov, I. A. Iakovlev, A. A. Iliasov, M. I. Katsnelson, and V. V. Mazurenko, Multiscale structural complexity of natural patterns, Proc. Natl. Acad. Sci. USA 117, 30241 (2020).
  46. S. T. Chan and E. Fried, Structural stability and thermodynamics of artistic composition, Proc. Natl. Acad. Sci. USA 121, e2406735121 (2024).

Outline

Information

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation