- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Typical reconstruction limit and phase transition of maximum entropy method

Masaru Hitomi1 and Masayuki Ohzeki1,2,3,4

  • 1Graduate School of Information Sciences, Tohoku University, Sendai 980-8579, Japan
  • 2Department of Physics, Institute of Science Tokyo, Tokyo 152-8551, Japan
  • 3Research and Education Institute for Semiconductors and Informatics, Kumamoto University, Kumamoto, Kumamoto 860-8555, Japan
  • 4Sigma-i Co., Ltd., Minato-ku, Tokyo 108-0075, Japan

PDF

Phys. Rev. Research 7, 043091 – Published 23 October, 2025

DOI: https://doi.org/10.1103/bg58-lpt2

Abstract

We investigate the dependence of the maximum entropy method (MEM) reconstruction performance on the default model. The maximum entropy method is a reconstruction technique that utilizes prior information, referred to as the default model, to recover original signals from observed data, and it is widely used in underdetermined systems. The broad applications have been reported in fields ranging from the analysis of observational data in seismology and astronomy, to large-scale computations in quantum chemistry, and even in social sciences such as linguistics. However, a known drawback of MEM is that its results depend on the assumed default model. In this study, we employ the replica method to elucidate how discrepancies in the default model affect the reconstruction of signals with specific distributions. We report that for binary signals, even small discrepancies can induce phase transitions, leading to reconstruction failure. Additionally, by comparing MEM with reconstruction based on L1-norm regularization, a method proposed in recent years, we demonstrate that MEM exhibits lower reconstruction accuracy under certain conditions.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (50)

  1. J. P. Burg, Maximum entropy spectral analysis, in Proceedings of 37th Meeting, Society of Exploration Geophysics, Oklahoma City (1967).
  2. J. P. Burg, A new analysis technique for time series data, in Advanced Study Institute on Signal Proceeding (Enschede, Netherlands, 1968).
  3. J. P. Burg, Maximum Entropy Spectral Analysis (Stanford University, Stanford, CA, 1975).
  4. J. P. Burg, The relationship between maximum entropy spectra and maximum likelihood spectra, Geophysics 37, 375 (1972).
  5. N. Andersen, On the calculation of filter coefficients for maximum entropy spectral analysis, Geophysics 39, 69 (1974).
  6. H. R. Radoski, P. F. Fougere, and E. J. Zawalick, A comparison of power spectral estimates and applications of the maximum entropy method, J. Geophys. Res. 80, 619 (1975).
  7. A. Jackson, C. Constable, and N. Gillet, Maximum entropy regularization of the geomagnetic core field inverse problem: Maximum entropy regularization, Geophys. J. Int. 171, 995 (2007).
  8. I. Zoukaneri and M. J. Porsani, A combined Wigner-Ville and maximum entropy method for high-resolution time-frequency analysis of seismic data, Geophysics 80, O1 (2015).
  9. B. Li, B. Liu, K. Guo, C. Li, and B. Wang, Application of a maximum entropy model for mineral prospectivity maps, Minerals 9, 556 (2019).
  10. Y. Wang, Y. Rao, and D. Xu, Multichannel maximum-entropy method for the Wigner-Ville distribution, Geophysics 85, V25 (2020).
  11. A. Martini, S. Schmidt, G. Ashton, and W. D. Pozzo, Maximum entropy spectral analysis: an application to gravitational waves data analysis, Eur. Phys. J. C 84, 1023 (2024).
  12. R. N. Silver, D. S. Sivia, and J. E. Gubernatis, Maximum-entropy method for analytic continuation of quantum Monte Carlo data, Phys. Rev. B 41, 2380 (1990).
  13. J. E. Gubernatis, M. Jarrell, R. N. Silver, and D. S. Sivia, Quantum Monte Carlo simulations and maximum entropy: Dynamics from imaginary-time data, Phys. Rev. B 44, 6011 (1991).
  14. W. von der Linden, M. Donath, and V. Dose, Unbiased access to exchange splitting of magnetic bands using the maximum entropy method, Phys. Rev. Lett. 71, 899 (1993).
  15. D. Bergeron and A.-M. S. Tremblay, Algorithms for optimized maximum entropy and diagnostic tools for analytic continuation, Phys. Rev. E 94, 023303 (2016).
  16. X.-H. Sui, H.-T. Wang, H. Tang, and Z.-B. Su, Quasiparticle density of states by inversion with maximum entropy method, Phys. Rev. B 94, 144505 (2016).
  17. G. J. Kraberger, R. Triebl, M. Zingl, and M. Aichhorn, Maximum entropy formalism for the analytic continuation of matrix-valued Green’s functions, Phys. Rev. B 96, 155128 (2017).
  18. K. Ghanem and E. Koch, Generalized maximum entropy methods as limits of the average spectrum method, Phys. Rev. B 108, L201107 (2023).
  19. Wernecke and D'Addario, Maximum entropy image reconstruction, IEEE Trans. Comput. C-26, 351 (1977).
  20. S. F. Gull and G. J. Daniell, Image reconstruction from incomplete and noisy data, Nature (London) 272, 686 (1978).
  21. S. F. Gull and J. Skilling, Maximum entropy method in image processing, in IEE Proceedings F (Communications, Radar and Signal Processing) (IET, London, 1984), Vol. 131, pp. 646–659.
  22. J. F. Martin, The maximum entropy method in NMR, J. Magn. Reson. 65, 291 (1985).
  23. E. G. Bazulin, The maximum entropy method in ultrasonic non-destructive testing—Increasing the resolution, image noise reduction and echo acquisition rate, Entropy 20, 621 (2018).
  24. F. Benedetto, G. Giunta, and L. Mastroeni, A maximum entropy method to assess the predictability of financial and commodity prices, Digital Signal Process. 46, 19 (2015).
  25. S. Kawahara, H. Katsuda, and G. Kumagai, Accounting for the stochastic nature of sound symbolism using maximum entropy model, Open Linguist. 5, 109 (2019).
  26. S. S. Shih, Gradient categories in lexically-conditioned phonology: An example from sound symbolism, in Proceedings of the Annual Meetings on Phonology (Linguistic Society of America, Washington, DC, 2019).
  27. K. L. Bouman et al., Computational imaging for VLBI image reconstruction, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, Piscataway, NJ, 2016), p. 913.
  28. K. Akiyama et al., Imaging the Schwarzschild-radius-scale Sructure of M87 with the Event Horizon Telescope using sparse modeling, Astrophys. J. 838, 1 (2017).
  29. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. I. The shadow of the supermassive black hole, Astrophys. J. Lett. 875, L1 (2019).
  30. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. II. Array and instrumentation, Astrophys. J. Lett. 875, L2 (2019).
  31. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. III. Data processing and calibration, Astrophys. J. Lett. 875, L3 (2019).
  32. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. IV. Imaging the central supermassive black hole, Astrophys. J. Lett. 875, L4 (2019).
  33. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. V. Physical origin of the asymmetric ring, Astrophys. J. Lett. 875, L5 (2019).
  34. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. VI. The shadow and mass of the central black hole, Astrophys. J. Lett. 875, L6 (2019).
  35. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. VII. Polarization of the ring, Astrophys. J. Lett. 910, L12 (2021).
  36. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. VIII. Magnetic field structure near the event horizon, Astrophys. J. Lett. 910, L13 (2021).
  37. K. Akiyama et al. (The Event Horizon Telescope Collaboration), First M87 Event Horizon Telescope results. IX. Detection of near-horizon circular polarization, Astrophys. J. Lett. 957, L20 (2023).
  38. D. L. Donoho, Compressed sensing, IEEE Trans. Inf. Theory 52, 1289 (2006).
  39. E. J. Candes and M. B. Wakin, An introduction to compressive sampling, IEEE Signal Process. Mag. 25, 21 (2008).
  40. D. L. Donoho and J. Tanner, Sparse nonnegative solution of underdetermined linear equations by linear programming, Proc. Natl. Acad. Sci. USA 102, 9452 (2005).
  41. E. J. Candes and T. Tao, Decoding by linear programming, IEEE Trans. Inf. Theory 51, 4203 (2005).
  42. E. J. Candes and T. Tao, Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406 (2006).
  43. E. J. Candes, The restricted isometry property and its implications for compressed sensing, C.R. Math. 346, 589 (2008).
  44. D. Angelosante, G. B. Giannakis, and E. Grossi, Compressed sensing of time-varying signals, in 16th International Conference on Digital Signal Processing (DSP 2009) (IEEE, Piscataway, NJ, 2009), pp. 1–8.
  45. C. G. Graff and E. Y. Sidky, Compressive sensing in medical imaging, Appl. Opt. 54, C23 (2015).
  46. Y. Kabashima, T. Wadayama, and T. Tanaka, A typical reconstruction limit for compressed sensing based on Lp-norm minimization, J. Stat. Mech. (2009) L09003.
  47. F. Krzakala, M. Mezard, F. Sausset, Y. Sun, and L. Zdeborova, Statistical-physics-based reconstruction in compressed sensing, Phys. Rev. X 2, 021005 (2012).
  48. S. Keiper, G. Kutyniok, D. G. Lee, and Pfander Götz, Compressed sensing for finite-dimensional subspaces, Linear Algebra Appl. 532, 570 (2017).
  49. M. Doi and M. Ohzeki, Phase transition in binary compressed sensing based on L1-norm minimization, J. Phys. Soc. Jpn. 93, 084003 (2024).
  50. M. Saito, Possible instability in the Burg maximum entropy method, J. Phys. Earth 26, 123 (1978).

Outline

Information

Phys. Rev. Research 7, 043091– Published 23 October, 2025
  • Received 9 April 2025
  • Accepted 26 September 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation