Export citation

Export citation

Choose format for download:

Download Citation

    Data compression with noise suppression for inference under noisy covariance

    Sunao Sugiyama* and Minsu Park

    • *Contact author: ssunao@sas.upenn.edu

    Phys. Rev. D 112, 123505 – Published 2 December, 2025

    DOI: https://doi.org/10.1103/hksr-253f

    Abstract

    In many fields including cosmology, statistical inference often relies on Gaussian likelihoods whose covariance matrices are estimated from a finite number of simulations. This finite-sample estimation introduces noise into the covariance, which propagates to parameter estimates, a phenomenon known as the Dodelson-Schneider (DS) effect, leading to inflated uncertainties. While the Massively optimized parameter estimation and data compression (MOPED) algorithm offers lossless Fisher information-preserving compression, it does not mitigate the DS effect when the compression matrix itself is derived from noisy covariances. In this paper, we propose a modified compression scheme, powered MOPED (p-MOPED), which suppresses noise propagation by balancing information retention and covariance estimate noise reduction through a tunable power-law transformation of the sample correlation matrix. We test p-MOPED against standard and diagonal MOPED on toy models and on cosmological data from the Subaru Hyper Suprime-Cam Year 3 weak lensing survey. Our results demonstrate that p-MOPED consistently outperforms other approaches, especially in regimes with limited simulations, offering a robust compression strategy for high-dimensional data analyses under practical constraints.

    Physics Subject Headings (PhySH)

    Authorization Required

    We need you to provide your credentials before accessing this content.

    References (Subscription Required)

    Outline

    Information

    Sign In to Your Journals Account

    Filter

    Filter

    Article Lookup

    Enter a citation