- Physical Review Journals
All JournalsPhysics Magazine

Physical Review E

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Distance function for stochastic matrices

Antony R. Lee* and Peter Tiňo

Iain B. Styles

  • School of Electronics, Electrical Engineering, and Computer Science, Queen's University Belfast, University Road, Belfast BT7 1NN, United Kingdom

  • *Contact author: arl290@student.bham.ac.uk
PDF

Phys. Rev. E 114, 014125 – Published 15 July, 2026

DOI: https://doi.org/10.1103/z34j-grq5

Abstract

The ability to quantify the similarity of stochastic processes is important in a wide range of areas including communications, simulation, and operational research. Such processes are commonly modeled as Markov Chains and so a natural way to compare their similarity is to compute distances between pairs of Markov Chains. We propose a distance function on the space of stochastic matrices that draws on ideas from information geometry. We first show that the Bhattacharyya angle is an appropriate measure of distance between Markov chain sequences, drawing on examples taken from healthcare processes and deriving bounds on the convergence of the distance and mixing times. We then extend these ideas to derive a distance measure which enables direct comparison of the transition matrices themselves. Our result is a true metric which has a closed form and is efficient to implement for numerical evaluation. In the case of ergodic Markov chains, it is shown that considering either the Bhattacharyya angle on Markov sequences or our stochastic matrix distance leads to the same distance between models.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (61)

  1. A. S. Omer and D. H. Woldegebreal, in Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, Vol. 443 (LNICST, London, UK, 2022).
  2. M. Abu Alsheikh, D. T. Hoang, D. Niyato, H. P. Tan, and S. Lin, Markov decision processes with applications in wireless sensor networks: A survey, IEEE Commun. Surv. Tutorials 17, 1239 (2015).
  3. V. Roy, Convergence diagnostics for Markov chain Monte Carlo, Annu. Rev. Stat. Appl. 7, 387 (2020).
  4. M. M. Hamdi, H. F. Mahdi, M. S. Abood, R. Q. Mohammed, A. D. Abbas, and A. H. Mohammed, A review on queue management algorithms in large networks, IOP Conf. Ser.: Mater. Sci. Eng. 1076, 012034 (2021).
  5. B. Mor, S. Garhwal, and A. Kumar, A systematic review of hidden Markov models and their applications, Arch. Comput. Methods Eng. 28, 1429 (2021).
  6. S. J. van Zelst, F. Mannhardt, M. de Leoni, and A. Koschmider, Event abstraction in process mining: Literature review and taxonomy, Granular Computing 6, 719 (2021).
  7. L. D. Valdez, L. Vassallo, and L. A. Braunstein, Epidemic control in networks with cliques, Phys. Rev. E 107, 054304 (2023).
  8. Y. Guo, L. Tu, H. Shen, and L. Chai, Transmission dynamics of disease spreading in multilayer networks with mass media, Phys. Rev. E 106, 034307 (2022).
  9. D. H. Silva, F. A. Rodrigues, and S. C. Ferreira, Accuracy of discrete- and continuous-time mean-field theories for epidemic processes on complex networks, Phys. Rev. E 110, 014302 (2024).
  10. M. A. Achterberg, B. Prasse, and P. Van Mieghem, Analysis of continuous-time Markovian ɛ-SIS epidemics on networks, Phys. Rev. E 105, 054305 (2022).
  11. N. Martin, G. Van Houdt, and G. Janssenswillen, DaQAPO: Supporting flexible and fine-grained event log quality assessment, Expert Syst. Appl. 191, 116274 (2022).
  12. E. De Roock and N. Martin, Process mining in healthcare—An updated perspective on the state of the art, J. Biomed. Inform. 127, 103995 (2022).
  13. J. Munoz-Gama, N. Martin, C. Fernandez-Llatas, O. A. Johnson, M. Sepúlveda, E. Helm, V. Galvez-Yanjari, E. Rojas, A. Martinez-Millana, D. Aloini, I. A. Amantea, R. Andrews, M. Arias, I. Beerepoot, E. Benevento, A. Burattin, D. Capurro, J. Carmona, M. Comuzzi, B. Dalmas, et al., Process mining for healthcare: Characteristics and challenges, J. Biomed. Inform. 127, 103994 (2022).
  14. D. Claudio, S. Moyce, T. Albano, E. Ibe, N. Miller, and M. O'Leary, A Markov chain model for mental health interventions, Int. J. Environ. Res. Public Healths 20, 3525 (2023).
  15. S. McClean, M. Faddy, and P. Millard, in Proceedings of the IEEE Symposium on Computer-Based Medical Systems (IEEE, Piscataway, NJ, 2006).
  16. B. A. Craig and P. P. Sendi, Estimation of the transition matrix of a discrete-time Markov chain, Health Economics 11, 33 (2002).
  17. T. Srivastava, N. R. Latimer, and P. Tappenden, Estimation of transition probabilities for state-transition models: A review of NICE appraisals, PharmacoEconomics 39, 869 (2021).
  18. Peginterferon beta-1a for treating relapsing–remitting multiple sclerosis, Tech. Rep. NICE (2020), https://www.nice.org.uk/guidance/ta624.
  19. R. Gatta, J. Lenkowicz, M. Vallati, E. Rojas, A. Damiani, L. Sacchi, B. De Bari, A. Dagliati, C. Fernandez-Llatas, M. Montesi, A. Marchetti, M. Castellano, and V. Valentini, in Artificial Intelligence in Medicine (AIME 2017), Lecture Notes in Computer Science LNAI, Vol. 10259 (Springer, Cham, 2017), pp. 351–355.
  20. F. Marazza, F. A. Bukhsh, O. Vijlbrief, J. Geerdink, S. Pathak, M. van Keulen, and C. Seifert, in Business Process Management Workshops (BPM 2019), Lecture Notes in Business Information Processing, LNBIP, Vol. 362 (Springer, Cham, 2019), pp. 496–507.
  21. A. F. Ghahfarokhi, A. Berti, and W. M. P. van der Aalst, Process comparison using object-centric process cubes, arXiv:2103.07184.
  22. M. Vallati, S. Orini, M. Lorusso, M. Savino, R. Gatta, and M. Filosto, On the comparison of Markov chains-based models in process mining for healthcare: A case study, in Proceedings of the 36th International FLAIRS Conference (LibraryPress@UF, 2023).
  23. V. M. Panaretos and Y. Zemel, Statistical aspects of Wasserstein distances, Annu. Rev. Stat. Appl. 6, 405 (2019).
  24. R. B. Lyngsø, C. N. Pedersen, and H. Nielsen, in Proceedings of the 7th International Conference on Intelligent Systems for Molecular Biology (ISMB'99) (AAAI Press, 1999), pp. 178–186.
  25. M. Jaeger, H. Mao, K. Guldstrand Larsen, and R. Mardare, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, Vol. 8657 (Springer International Publishing, Berlin, 2014), pp. 297–312.
  26. P. Daca, T. A. Henzinger, J. Křetínský, and T. Petrov, Linear distances between Markov chains, in 27th International Conference on Concurrency Theory (CONCUR 2016), Leibniz International Proceedings in Informatics (LIPIcs) (Schloss Dagstuhl - LZI GmbH, 2016), Vol. 59.
  27. S. I. Amari, Information geometry on hierarchy of probability distributions, IEEE Trans. Inf. Theory 47, 1701 (2001).
  28. S. i. Amari, Information geometry, Jpn. J. Math. 16, 1 (2021).
  29. J. E. Cohen, J. H. B. Kemperman, and G. Zbăganu, Comparisons of Stochastic Matrices with Applications in Information Theory, Statistics, Economics and Population Sciences, 1st ed. (Birkhäuser, Boston, MA, 1998).
  30. Y. Zhou, Q. Yuan, C. Yang, and Y. Wang, Who you are determines how you travel: Clustering human activity patterns with a Markov-chain-based mixture model, Travel Behav. Soc. 24, 102 (2021).
  31. J. Sanders, A. Proutière, and S.-Y. Yun, Clustering in block Markov chains, Ann. Stat. 48, 3488 (2020).
  32. T. Kawabata and K. Nishikawa, Protein structure comparison using the Markov transition model of evolution, Proteins: Struct. Funct. Genet. 41, 108 (2000).
  33. R. He and J. McAuley, in Proceedings of the IEEE International Conference on Data Mining (IEEE, Piscataway, NJ, ICDM'16) (2016).
  34. C. Wang, B. Wang, H. Liang, J. Zhang, W. Huang, and W. Zhang, A weighted transition matrix learning algorithm for the sensor-based human activity recognition, IEEE Access 8, 72870 (2020).
  35. Y. Yao, T. Liu, B. Han, M. Gong, J. Deng, G. Niu, and M. Sugiyama, Dual T: Reducing estimation error for transition matrix in label-noise learning, arXiv:2006.07805.
  36. Y. Zou, T. Liu, D. Liu, and F. Sun, Reinforcement learning-based real-time energy management for a hybrid tracked vehicle, Appl. Energy 171, 372 (2016).
  37. G. Du, Y. Zou, X. Zhang, Z. Kong, J. Wu, and D. He, Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning, Appl. Energy 251, 113388 (2019).
  38. J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, Structured denoising diffusion models in discrete state-spaces, in Proceedings of the 35th International Conference on Neural Information Processing System, NIPS'21 (Curran Associates Inc., Red Hook, NY, 2021).
  39. A. A. Jabri, A. Owens, and A. A. Efros, Space-time correspondence as a contrastive random walk, arXiv:2006.14613.
  40. I. Nevat, D. M. Divakaran, S. G. Nagarajan, P. Zhang, L. Su, L. L. Ko, and V. L. Thing, Anomaly detection and attribution in networks with temporally correlated traffic, IEEE/ACM Trans. Netw. 26, 131 (2018).
  41. S. Bacallado, J. D. Chodera, and V. Pande, Bayesian comparison of Markov models of molecular dynamics with detailed balance constraint, J. Chem. Phys. 131, 045106 (2009).
  42. C. C. Strelioff, J. P. Crutchfield, and A. W. Hübler, Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling, Phys. Rev. E 76, 011106 (2007).
  43. G. Wolfer and S. Watanabe, Information geometry of Markov kernels: A survey, Front. Phys. 11, 1195562 (2023).
  44. S. Kiefer, On computing the total variation distance of hidden Markov models, in 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018), Leibniz International Proceedings in Informatics (LIPIcs) (Schloss Dagstuhl, 2018), Vol. 107.
  45. F. Nielsen, An elementary introduction to information geometry, Entropy 22, 1100 (2020).
  46. H. K. Miyamoto, F. C. C. Meneghetti, J. Pinele, and S. I. R. Costa, On closed-form expressions for the Fisher–Rao distance, Inf. Geom. 7, 311 (2024).
  47. A. Bhattacharyya, On a measure of divergence between two multinomial populations, Indian J. Stat. (1933-1960) 7, 401 (1946).
  48. F. J. Aherne, N. A. Thacker, and P. I. Rockett, The Bhattacharyya metric as an absolute similarity measure for frequency coded data, Kybernetika 34, 363 (1998).
  49. S. Bi, M. Broggi, and M. Beer, The role of the Bhattacharyya distance in stochastic model updating, Mech. Syst. Sig. Process. 117, 437 (2019).
  50. D. Kazakos, The Bhattacharyya distance and detection between Markov chains, IEEE Trans. Inf. Theory 24, 747 (1978).
  51. Z. Rached, F. Alajaji, and L. Campbell, The Kullback–Leibler divergence rate between Markov sources, IEEE Trans. Inf. Theory 50, 917 (2004).
  52. M. A. Munem and D. J. Foulis, Calculus with Analytical Geometry, 2nd ed. (W. H. Freeman, New York, NY, 1984), p. 1048.
  53. Z. Huang, On the spectral radius and the spectral norm of Hadamard products of nonnegative matrices, Linear Algebra Appl. 434, 457 (2011).
  54. R. Montenegro and P. Tetali, Mathematical aspects of mixing times in Markov chains, Found. Trends Theor. Comput. Sci. 1, 237 (2006).
  55. M. Dyer, L. A. Goldberg, M. Jerrum, and R. Martin, Markov chain comparison, Probab. Surv. 3, 89 (2006).
  56. G. Lebanon, Axiomatic geometry of conditional models, IEEE Trans. Inf. Theory 51, 1283 (2005).
  57. G. Montúfar, J. Rauh, and N. Ay, On the Fisher metric of conditional probability polytopes, Entropy 16, 3207 (2014).
  58. D. Steinley, Properties of the Hubert-Arable adjusted Rand index, Psychol. Methods 9, 386 (2004).
  59. T. Chen and C. Guestrin, in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, New York, NY, 2016).
  60. https://github.com/AntonyRLee/stochastic_matrix_distance.
  61. D. Dembélé, A method for computing the Perron root for primitive matrices, Numer. Linear Algebra Appl. 28, e2340 (2021).

Outline

Information

Phys. Rev. E 114, 014125– Published 15 July, 2026
  • Received 6 May 2025
  • Revised 4 March 2026
  • Accepted 1 June 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation