- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Calibrating Bayesian tension statistics using neural ratio estimators

Harry T. J. Bevins1,2,*, William J. Handley2,3, and Thomas Gessey-Jones1,2

  • 1Astrophysics Group, Cavendish Laboratory, Cambridge, CB3 0HE, United Kingdom
  • 2Kavli Institute for Cosmology, Cambridge, CB3 0HA, United Kingdom
  • 3Institute of Astronomy, Cambridge, CB3 0HA, United Kingdom

  • *Contact author: htjb2@cam.ac.uk
PDF

Phys. Rev. D 112, 043520 – Published 15 August, 2025

DOI: https://doi.org/10.1103/47hn-h8kc

Abstract

When fits of the same physical model to two different datasets disagree, we call this tension. Several apparent tensions in cosmology have occupied researchers in recent years, and a number of different metrics have been proposed to quantify tension. Many of these metrics suffer from limiting assumptions, and correctly calibrating these is essential if we want to successfully determine whether discrepancies are significant. A commonly used metric of tension is the evidence ratio R. The statistic has been widely adopted by the community as a Bayesian way of quantifying tensions, however, it has a nontrivial dependence on the prior that is not always accounted for properly. We show that this can be calibrated out effectively with neural ratio estimation. We demonstrate our proposed calibration technique with an analytic example, a toy example inspired by 21-cm cosmology, and with observations of the baryon acoustic oscillations from the Dark Energy Spectroscopic Instrument (DESI) and the Sloan Digital Sky Survey (SDSS). We find no significant tension between DESI and SDSS.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (81)

  1. L. Knox and M. Millea, Hubble constant hunter’s guide, Phys. Rev. D 101, 043533 (2020).
  2. G. Efstathiou, A lockdown perspective on the Hubble tension (with comments from the SH0ES team), arXiv:2007.10716.
  3. A. Amon and G. Efstathiou, A non-linear solution to the S8 tension?, Mon. Not. R. Astron. Soc. 516, 5355 (2022).
  4. C. Preston, A. Amon, and G. Efstathiou, A non-linear solution to the S8 tension—II. Analysis of DES Year 3 cosmic shear, Mon. Not. R. Astron. Soc. 525, 5554 (2023).
  5. Dark Energy Survey and Kilo-Degree Survey Collaboration, DES Y3 + KiDS-1000: Consistent cosmology combining cosmic shear surveys, Open J. Astrophys. 6, 36 (2023).
  6. S. Singh, N. T. Jishnu, R. Subrahmanyan, N. Udaya Shankar, B. S. Girish, A. Raghunathan, R. Somashekar, K. S. Srivani, and M. Sathyanarayana Rao, On the detection of a cosmic dawn signal in the radio background, Nat. Astron. 6, 607 (2022).
  7. R. A. Battye, T. Charnock, and A. Moss, Tension between the power spectrum of density perturbations measured on large and small scales, Phys. Rev. D 91, 103508 (2015).
  8. W. Handley, Curvature tension: Evidence for a closed universe, Phys. Rev. D 103, L041301 (2021).
  9. P. J. E. Peebles, Tests of cosmological models constrained by inflation, Astrophys. J. 284, 439 (1984).
  10. L. M. Krauss and M. S. Turner, The cosmological constant is back, Gen. Relativ. Gravit. 27, 1137 (1995).
  11. J. P. Ostriker and P. J. Steinhardt, The observational case for a low-density Universe with a non-zero cosmological constant, Nature (London) 377, 600 (1995).
  12. G. Efstathiou, W. J. Sutherland, and S. J. Maddox, The cosmological constant and cold dark matter, Nature (London) 348, 705 (1990).
  13. A. G. Riess et al., Observational evidence from supernovae for an accelerating universe and a cosmological constant, Astron. J. 116, 1009 (1998).
  14. E. Abdalla et al., Cosmology intertwined: A review of the particle physics, astrophysics, and cosmology associated with the cosmological tensions and anomalies, J. High Energy Astrophys. 34, 49 (2022).
  15. W. Handley and P. Lemos, Quantifying tensions in cosmological parameters: Interpreting the DES evidence ratio, Phys. Rev. D 100, 043504 (2019).
  16. M. Raveri, G. Zacharegkas, and W. Hu, Quantifying concordance of correlated cosmological data sets, Phys. Rev. D 101, 103527 (2020).
  17. M. Raveri and C. Doux, Non-Gaussian estimates of tensions in cosmological parameters, Phys. Rev. D 104, 043504 (2021).
  18. M. Raveri and W. Hu, Concordance and discordance in cosmology, Phys. Rev. D 99, 043506 (2019).
  19. Y. Park and E. Rozo, Concordance cosmology?, Mon. Not. R. Astron. Soc. 499, 4638 (2020).
  20. T. Charnock, R. A. Battye, and A. Moss, Planck data versus large scale structure: Methods to quantify discordance, Phys. Rev. D 95, 123535 (2017).
  21. DES Collaboration, Assessing tension metrics with dark energy survey and Planck data, Mon. Not. R. Astron. Soc. 505, 6179 (2021).
  22. E. Saraivanov, K. Zhong, V. Miranda, S. S. Boruah, T. Eifler, and E. Krause, Attention-based neural network emulators for multi-probe data vectors Part II: Assessing tension metrics, Phys. Rev. D 111, 123520 (2025).
  23. P. Marshall, N. Rajguru, and A. Slosar, Bayesian evidence as a tool for comparing datasets, Phys. Rev. D 73, 067302 (2006).
  24. R. Trotta, Bayes in the sky: Bayesian inference and model selection in cosmology, Contemp. Phys. 49, 71 (2008).
  25. S. Seehars, S. Grandis, A. Amara, and A. Refregier, Quantifying concordance in cosmology, Phys. Rev. D 93, 103507 (2016).
  26. M. Cortês and A. R. Liddle, On data set tensions and signatures of new cosmological physics, Mon. Not. R. Astron. Soc. 531, L52 (2024).
  27. A. Cuceu, J. Farr, P. Lemos, and A. Font-Ribera, Baryon Acoustic Oscillations and the Hubble constant: Past, present and future, J. Cosmol. Astropart. Phys. 10 (2019) 044.
  28. K. Cranmer, J. Brehmer, and G. Louppe, The frontier of simulation-based inference, Proc. Natl. Acad. Sci. U.S.A. 117, 30055 (2020).
  29. B. Miller, A. Cole, P. Forré, G. Louppe, and C. Weniger, Truncated marginal neural ratio estimation, Adv. Neural Inf. Process. Syst. 34, 129 (2021), https://proceedings.neurips.cc/paper/2021/hash/01632f7b7a127233fa1188bd6c2e42e1-Abstract.html.
  30. A. Cole, B. K. Miller, S. J. Witte, M. X. Cai, M. W. Grootes, F. Nattino, and C. Weniger, Fast and credible likelihood-free cosmology with truncated marginal neural ratio estimation, J. Cosmol. Astropart. Phys. 09 (2022) 004.
  31. J. Skilling, Nested sampling for general Bayesian computation, Bayesian Anal. 1, 833 (2006).
  32. G. Ashton et al., Nested sampling for physical scientists, Nat. Rev. Methods Primers 2, 39 (2022).
  33. R. Trotta, Applications of Bayesian model selection to cosmological parameters, Mon. Not. R. Astron. Soc. 378, 72 (2007).
  34. A. Heavens, Y. Fantaye, A. Mootoovaloo, H. Eggers, Z. Hosenie, S. Kroon, and E. Sellentin, Marginal likelihoods from Monte Carlo Markov chains, arXiv:1704.03472.
  35. R. Srinivasan, M. Crisostomi, R. Trotta, E. Barausse, and M. Breschi, floZ: Evidence estimation from posterior samples with normalizing flows, Phys. Rev. D 110, 123007 (2024).
  36. D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman, emcee: The MCMC Hammer, Publ. Astron. Soc. Pac. 125, 306 (2013).
  37. A. Polanska, M. A. Price, A. Spurio Mancini, and J. D. McEwen, Learned harmonic mean estimation of the marginal likelihood with normalizing flows, arXiv:2307.00048.
  38. Dark Energy Survey Collaboration, Dark Energy Survey year 1 results: Cosmological constraints from galaxy clustering and weak lensing, Phys. Rev. D 98, 043526 (2018).
  39. H. Jeffreys, Theory of Probability, International Series of Monographs on Physics (Clarendon Press, New York, 1983).

  40. https://github.com/handley-lab/lsbi.

  41. S. R. Furlanetto, S. P. Oh, and F. H. Briggs, Cosmology at low frequencies: The 21 cm transition and the high-redshift Universe, Phys. Rep. 433, 181 (2006).
  42. R. Barkana, The rise of the first stars: Supersonic streaming, radiative feedback, and 21-cm cosmology, Phys. Rep. 645, 1 (2016).
  43. The Cosmic 21-cm Revolution, edited by A. Mesinger (IOP Publishing, 2019), pp. 2514–3433.
  44. A. Liu and J. R. Shaw, Data analysis for precision 21 cm cosmology, Publ. Astrob. Soc. Pac. 132, 062001 (2020).
  45. J. D. Bowman, A. E. E. Rogers, R. A. Monsalve, T. J. Mozdzen, and N. Mahesh, An absorption profile centred at 78 megahertz in the sky-averaged spectrum, Nature (London) 555, 67 (2018).
  46. S. Singh, N. T. Jishnu, R. Subrahmanyan, N. Udaya Shankar, B. S. Girish, A. Raghunathan, R. Somashekar, K. S. Srivani, and M. Sathyanarayana Rao, On the detection of a cosmic dawn signal in the radio background, Nat. Astron. 6, 607 (2022).
  47. E. de Lera Acedo et al., The REACH radiometer for detecting the 21-cm hydrogen signal from redshift z≈7.5−28, Nat. Astron. 6, 984 (2022).
  48. J. Mirocha, Decoding the x-ray properties of pre-reionization era sources, Mon. Not. R. Astron. Soc. 443, 1211 (2014).
  49. A. Mesinger, S. Furlanetto, and R. Cen, 21 cmfast: A fast, seminumerical simulation of the high-redshift 21-cm signal, Mon. Not. R. Astron. Soc. 411, 955 (2011).
  50. I. Reis, A. Fialkov, and R. Barkana, High-redshift radio galaxies: A potential new source of 21-cm fluctuations, Mon. Not. R. Astron. Soc. 499, 5993 (2020).
  51. I. Reis, A. Fialkov, and R. Barkana, The subtlety of Ly α photons: Changing the expected range of the 21-cm signal, Mon. Not. R. Astron. Soc. 506, 5479 (2021).
  52. T. Gessey-Jones, N. S. Sartorio, A. Fialkov, G. M. Mirouh, M. Magg, R. G. Izzard, E. de Lera Acedo, W. J. Handley, and R. Barkana, Impact of the primordial stellar initial mass function on the 21-cm signal, Mon. Not. R. Astron. Soc. 516, 841 (2022).
  53. S. Sikder, R. Barkana, A. Fialkov, and I. Reis, Strong 21-cm fluctuations and anisotropy due to the line-of-sight effect of radio galaxies at cosmic dawn, Mon. Not. R. Astron. Soc. 527, 10975 (2024).
  54. S. Pochinda, T. Gessey-Jones, H. T. J. Bevins, A. Fialkov, S. Heimersheim, I. Abril-Cabezas, E. de Lera Acedo, S. Singh, S. Sikder, and R. Barkana, Constraining the properties of Population III galaxies with multiwavelength observations, Mon. Not. R. Astron. Soc. 531, 1113 (2024).
  55. T. Gessey-Jones, S. Pochinda, H. T. J. Bevins, A. Fialkov, W. J. Handley, E. de Lera Acedo, S. Singh, and R. Barkana, On the constraints on superconducting cosmic strings from 21-cm cosmology, Mon. Not. R. Astron. Soc. 524, 519 (2024).
  56. J. B. Muñoz, An effective model for the cosmic-dawn 21-cm signal, Mon. Not. R. Astron. Soc. 523, 2587 (2023).
  57. S. Mittal, ECHO21: Exploring the cosmos with hydrogen observation, arXiv:2503.11762.
  58. D. Anstey, E. de Lera Acedo, and W. Handley, A general Bayesian framework for foreground modelling and chromaticity correction for global 21 cm experiments, Mon. Not. R. Astron. Soc. 506, 2041 (2021).
  59. H. T. J. Bevins, W. J. Handley, A. Fialkov, E. de Lera Acedo, L. J. Greenhill, and D. C. Price, MAXSMOOTH: Rapid maximally smooth function fitting with applications in global 21-cm cosmology, Mon. Not. R. Astron. Soc. 502, 4405 (2021).
  60. R. Hills, G. Kulkarni, P. D. Meerburg, and E. Puchwein, Concerns about modelling of the EDGES data, Nature (London) 564, E32 (2018).
  61. S. Singh and R. Subrahmanyan, The redshifted 21 cm signal in the EDGES low-band spectrum, Astrophys. J. 880, 26 (2019).
  62. P. H. Sims and J. C. Pober, Testing for calibration systematics in the EDGES low-band data using Bayesian model selection, Mon. Not. R. Astron. Soc. 492, 22 (2020).
  63. W. J. Handley, M. P. Hobson, and A. N. Lasenby, polychord: Next-generation nested sampling, Mon. Not. R. Astron. Soc. 453, 4384 (2015).
  64. W. J. Handley, M. P. Hobson, and A. N. Lasenby, polychord: Nested sampling for cosmology, Mon. Not. R. Astron. Soc. 450, L61 (2015).
  65. S. Alam et al., The eleventh and twelfth data releases of the Sloan Digital Sky Survey: Final data from SDSS-III, Astrophys. J. Suppl. Ser. 219, 12 (2015).
  66. R. Ahumada et al., The 16th data release of the Sloan Digital Sky Surveys: First release from the APOGEE-2 southern survey and full release of eBOSS spectra, Astrophys. J. Suppl. Ser. 249, 3 (2020).
  67. A. Adame, J. Aguilar, S. Ahlen, S. Alam, D. Alexander, M. Alvarez, O. Alves, A. Anand, U. Andrade, E. Armengaud et al., Desi 2024 VI: Cosmological constraints from the measurements of baryon acoustic oscillations, J. Cosmol. Astropart. Phys. 02 (2025) 021.
  68. B. Bassett and R. Hlozek, in Dark Energy: Observational and Theoretical Approaches, edited by P. Ruiz-Lapuente (Cambridge University Press, Cambridge, UK, 2010), p. 246; arXiv:0910.5224.
  69. A. Lewis, A. Challinor, and A. Lasenby, Efficient computation of CMB anisotropies in closed FRW models, Astrophys. J. 538, 473 (2000).
  70. A. Lewis and S. Bridle, Cosmological parameters from CMB and other data: A Monte Carlo approach, Phys. Rev. D 66, 103511 (2002).
  71. DESI Collaboration, DESI 2024 IV: Baryon acoustic oscillations from the Lyman Alpha Forest, J. Cosmol. Astropart. Phys. 01 (2025) 124.
  72. DESI Collaboration, DESI 2024 III: Baryon acoustic oscillations from galaxies and quasars, J. Cosmol. Astropart. Phys. 04 (2025) 012.
  73. Planck Collaboration, Planck 2018 results. VI. Cosmological parameters, Astron. Astrophys. 641, A6 (2020).

  74. https://cosmologist.info/cosmomc/.

  75. https://cobaya.readthedocs.io/en/latest/likelihood_bao.html.

  76. D. J. Watts et al., COSMOGLOBE DR1 results. I. Improved Wilkinson Microwave Anisotropy Probe maps through Bayesian end-to-end analysis, Astron. Astrophys. 679, A143 (2023).
  77. K. Karchev, M. Grayling, B. M. Boyd, R. Trotta, K. S. Mandel, and C. Weniger, SIDE-real: Supernova Ia dust extinction with truncated marginal neural ratio estimation applied to real data, Mon. Not. R. Astron. Soc. 530, 3881 (2024).
  78. P. Lemos, A. Coogan, Y. Hezaveh, and L. Perreault-Levasseur, in International Conference on Machine Learning (PMLR, 2023), pp. 19256–19273.
  79. CDF Collaboration, High-precision measurement of the W boson mass with the CDF II detector, Science 376, 170 (2022).
  80. www.dirac.ac.uk
  81. https://github.com/htjb/tension-networks

Outline

Information

Phys. Rev. D 112, 043520– Published 15 August, 2025
  • Received 22 July 2024
  • Accepted 21 July 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation