- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Description length of canonical and microcanonical models

Francesca Giuffrida1,2,*, Tiziano Squartini1,3,4, Peter Grünwald5,6, and Diego Garlaschelli1,2,3

  • *Contact author: francesca.giuffrida@imtlucca.it
PDF

Phys. Rev. Research 7, 043057 – Published 16 October, 2025

DOI: https://doi.org/10.1103/1sd2-rxmy

Abstract

The (non)equivalence of canonical and microcanonical ensembles is a fundamental question in statistical physics, concerning whether the use of soft and hard constraints in the maximum-entropy construction leads to the same description of a system. Despite the fact that maximum-entropy models are also commonly used in statistical inference, pattern detection, and hypothesis testing, a complete understanding of the effects of ensemble nonequivalence on statistical modeling is still missing. Here, we study this problem from a rigorous model selection perspective by comparing canonical and microcanonical models via the minimum description length principle, which yields a trade-off between likelihood, measuring model accuracy, and complexity, measuring model flexibility and its potential to overfit data. We compute the normalized maximum likelihood (NML) of both formulations and find that (1) microcanonical models always achieve higher likelihood but are always more complex; (2) the optimal model choice depends on the empirical values of the constraints—the canonical model performs best when its fit to the observed data exceeds its uniform average fit across all realizations; (3) in the thermodynamic limit, the difference in description length per node vanishes when ensemble equivalence holds but persists otherwise, showing that nonequivalence implies extensive differences between large canonical and microcanonical models. Finally, we compare the NML approach to Bayesian methods, showing that (4) the choice of priors, practically irrelevant in equivalent models, becomes crucial when an extensive number of constraints are enforced, possibly leading to very different outcomes.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (48)

  1. T. Squartini and D. Garlaschelli, Maximum-Entropy Networks: Pattern Detection, Network Reconstruction and Graph Combinatorics (Springer, Cham, 2017).
  2. G. Cimini, T. Squartini, F. Saracco, D. Garlaschelli, A. Gabrielli, and G. Caldarelli, The statistical physics of real-world networks, Nat. Rev. Phys. 1, 58 (2019).
  3. T. Squartini, G. Caldarelli, G. Cimini, A. Gabrielli, and D. Garlaschelli, Reconstruction methods for networks: The case of economic and financial systems, Phys. Rep. 757, 1 (2018).
  4. R. Marcaccioli and G. Livan, Maximum entropy approach to multivariate time series randomization, Sci. Rep. 10, 10656 (2020).
  5. R. Marcaccioli and G. Livan, Correspondence between temporal correlations in time series, inverse problems, and the spherical model, Phys. Rev. E 102, 012112 (2020).
  6. J. Gibbs, Elementary Principles in Statistical Mechanics: Developed with Especial Reference to the Rational Foundation of Thermodynamics, Cambridge Library Collection—Mathematics (Cambridge University Press, Cambridge, UK, 2010).
  7. M. Blume, V. J. Emery, and R. B. Griffiths, Ising model for the λ transition and phase separation in He3−He4 mixtures, Phys. Rev. A 4, 1071 (1971).
  8. D. Lynden-Bell, Negative specific heat in astronomy, physics and chemistry, Physica A 263, 293 (1999).
  9. R. S. Ellis, K. Haven, and B. Turkington, Large deviation principles and complete equivalence and nonequivalence results for pure and mixed ensembles, J. Stat. Phys. 101, 999 (2000).
  10. M. D’Agostino, F. Gulminelli, Ph. Chomaz, M. Bruno, F. Cannata, R. Bougault, F. Gramegna, I. Iori, N. Le Neindre, G. V. Margagliotti et al., Negative heat capacity in the critical region of nuclear fragmentation: An experimental evidence of the liquid-gas phase transition, Phys. Lett. B 473, 219 (2000).
  11. J. Barré, D. Mukamel, and S. Ruffo, Inequivalence of ensembles in a system with long-range interactions, Phys. Rev. Lett. 87, 030601 (2001).
  12. P. H. Chavanis, Gravitational instability of isothermal and polytropic spheres, Astron. Astrophys. 401, 15 (2003).
  13. R. S. Ellis, H. Touchette, and B. Turkington, Thermodynamic versus statistical nonequivalence of ensembles for the mean-field Blume–Emery–Griffiths model, Physica A 335, 518 (2004).
  14. F. Bouchet and J. Barré, Classification of phase transitions and ensemble inequivalence in systems with long range interactions, J. Stat. Phys. 118, 1073 (2005).
  15. J. Barré and B. Gonçalves, Ensemble inequivalence in random graphs, Physica A 386, 212 (2007).
  16. C. Radin and L. Sadun, Phase transitions in a complex network, J. Phys. A: Math. Theor. 46, 305002 (2013).
  17. A. Campa, T. Dauxois, D. Fanelli, and S. Ruffo, Equilibrium statistical mechanics of long-range interactions, in Physics of Long-Range Interacting Systems (Oxford University Press, Oxford, UK, 2014).
  18. H. Touchette, Equivalence and nonequivalence of ensembles: Thermodynamic, macrostate, and measure levels, J. Stat. Phys. 159, 987 (2015).
  19. T. Squartini, J. de Mol, F. den Hollander, and D. Garlaschelli, Breaking of ensemble equivalence in networks, Phys. Rev. Lett. 115, 268701 (2015).
  20. T. Squartini and D. Garlaschelli, Reconnecting statistical physics and combinatorics beyond ensemble equivalence, arXiv:1710.11422.
  21. Q. Zhang and D. Garlaschelli, Strong ensemble nonequivalence in systems with local constraints, New J. Phys. 24, 043011 (2022).
  22. D. Garlaschelli, F. den Hollander, and A. Roccaverde, Covariance structure behind breaking of ensemble equivalence in random graphs, J. Stat. Phys. 173, 644 (2018).
  23. M. Bruno, F. Saracco, D. Garlaschelli, C. J. Tessone, and G. Caldarelli, The ambiguity of nestedness under soft and hard constraints, Sci. Rep. 10, 19903 (2020).
  24. D. Garlaschelli and T. Squartini, Maximum-Entropy Networks: Pattern Detection, Network Reconstruction and Graph Combinatorics (Springer International Publishing, Cham, Switzerland, 2017).
  25. T. Squartini, R. Mastrandrea, and D. Garlaschelli, Unbiased sampling of network ensembles, New J. Phys. 17, 023052 (2015).
  26. D. Garlaschelli, F. den Hollander, and A. Roccaverde, Ensemble nonequivalence in random graphs with modular structure, J. Phys. A: Math. Theor. 50, 015001 (2017).
  27. H. Akaike, Information theory and an extension of the maximum likelihood principle, in Selected Papers of Hirotugu Akaike, edited by E. Parzen, K. Tanabe, and G. Kitagawa (Springer, New York, 1973), pp. 199–213.
  28. G. Schwarz, Estimating the dimension of a model, Ann. Statist. 6, 461 (1978).
  29. K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (Springer, New York, NY, 2010).
  30. P. Grünwald, The Minimum Description Length Principle (MIT Press, Cambridge, MA, 2007).
  31. P. Grünwald and T. Roos, Minimum description length revisited, Int. J. Math. Ind. 11, 1930001 (2019).
  32. K. Yamanishi, Learning with the Minimum Description Length Principle (Springer Nature, Singapore, 2023).
  33. J. J. Rissanen, Fisher information and stochastic complexity, IEEE Trans. Inf. Theory 42, 40 (1996).
  34. M. Li and P. Vitányi, An Introduction to Kolmogorov Complexity and Its Applications (Springer, New York, 2008), 3rd ed.
  35. E. T. Jaynes, Information theory and statistical mechanics, Phys. Rev. 106, 620 (1957).
  36. P. W. Holland and S. Leinhardt, An exponential family of probability distributions for directed graphs, J. Am. Stat. Assoc. 76, 33 (1981).
  37. D. R. Hunter and M. S. Handcock, Inference in curved exponential family models for networks, J. Comput. Graph. Stat. 15, 565 (2006).
  38. F. Saracco, M. J. Straka, R. Di Clemente, A. Gabrielli, G. Caldarelli, and T. Squartini, Inferring monopartite projections of bipartite networks: An entropy-based approach, New J. Phys. 19, 053022 (2017).
  39. P. P. A. Staniczenko, M. J. Smith, and S. Allesina, Selecting food web models using normalized maximum likelihood, Methods Ecol. Evol. 5, 551 (2014).
  40. T. P. Peixoto, Nonparametric Bayesian inference of the microcanonical stochastic block model, Phys. Rev. E 95, 012317 (2017).
  41. T. Vallès-Català, T. P. Peixoto, M. Sales-Pardo, and R. Guimerà, Consistencies and inconsistencies between model selection and link prediction in networks, Phys. Rev. E 97, 062316 (2018).
  42. R. E. Kass and A. E. Raftery, Bayes factors, J. Am. Stat. Assoc. 90, 773 (1995).
  43. H. Jeffreys, An invariant form for the prior probability in estimation problems, Proc. R. Soc. London A 186, 453 (1946).
  44. J. Rissanen, Strong optimality of the normalized ML models as universal codes and information in data, IEEE Trans. Inform. Theory 47, 1712 (2001).
  45. https://pnrr.sobigdata.it/.
  46. A. I. Khinchin, Mathematical Foundations of Information Theory (Dover Publications, New York, 1957).
  47. A. Somazzi and D. Garlaschelli, Learn your entropy from informative data: An axiom ensuring the consistent identification of generalized entropies, Phys. Rev. Res. 7, 033087 (2025).
  48. V. De Angelis, Stirling's series revisited, Am. Math. Mon. 116, 839 (2009).

Outline

Information

Phys. Rev. Research 7, 043057– Published 16 October, 2025
  • Received 26 November 2024
  • Accepted 30 July 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation