- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Training neural control variates using correlated configurations

Hyunwoo Oh*

  • Department of Physics and Maryland Center for Fundamental Physics, University of Maryland, College Park, Maryland 20742 USA

  • *Contact author: hyunwooh@umd.edu
PDF

Phys. Rev. D 112, 074501 – Published 7 October, 2025

DOI: https://doi.org/10.1103/kpc8-cj7p

Abstract

Neural control variates (NCVs) have emerged as a powerful tool for variance reduction in Monte Carlo (MC) simulations, particularly in high-dimensional problems where traditional control variates are difficult to construct analytically. By training neural networks to learn auxiliary functions correlated with the target observable, NCVs can significantly reduce estimator variance while preserving unbiasedness. However, a critical but often overlooked aspect of NCV training is the role of autocorrelated samples generated by Markov chain Monte Carlo (MCMC). While such samples are typically discarded for error estimation due to their statistical redundancy, they may contain useful information about the structure of the underlying probability distribution that can benefit the training process. In this work, we systematically examine the effect of using correlated configurations in training neural control variates. We demonstrate, both conceptually and numerically, that training on correlated data can improve control variate performance, especially in settings with limited computational resources. Our analysis includes empirical results from U(1) gauge theory and scalar field theory, illustrating when and how autocorrelated samples enhance NCV construction. These findings provide practical guidance for the efficient use of MCMC data in training neural networks.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (35)

  1. H. Kahn and A. W. Marshall, Methods of reducing sample size in Monte Carlo computations, J. Oper. Res. Soc. Am. 1, 263 (1953).
  2. E. C. Fieller and H. O. Hartley, Sampling with control variables, Biometrika 41, 494 (1954).
  3. P. F. Bedaque and H. Oh, Leveraging neural control variates for enhanced precision in lattice field theory, Phys. Rev. D 109, 094519 (2024).
  4. R. Wan, M. Zhong, H. Xiong, and Z. Zhu, Neural control variates for Monte Carlo variance reduction, in Machine Learning and Knowledge Discovery in Databases, edited by U. Brefeld, E. Fromont, A. Hotho, A. Knobbe, M. Maathuis, and C. Robardet (Springer International Publishing, Cham, 2020), pp. 533–547.
  5. Z. Sun, A. Barp, and F.-X. Briol, Vector-valued control variates, in Proceedings of the 40th International Conference on Machine Learning, ICML’23 (JMLR.org, Cambridge, MA, USA, 2023).
  6. Z. Sun, C. J. Oates, and F.-X. Briol, Meta-learning control variates: Variance reduction with limited data, in Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI ’23 (JMLR.org, Cambridge, MA, USA, 2023).
  7. K. Ott, M. Tiemann, P. Hennig, and F.-X. Briol, Bayesian numerical integration with neural networks, in Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI ’23 (JMLR.org, Cambridge, MA, USA, 2023).
  8. W. Grathwohl, D. Choi, Y. Wu, G. Roeder, and D. Duvenaud, Backpropagation through the void: Optimizing control variates for black-box gradient estimation, in International Conference on Learning Representations (Amherst, Massachusetts, USA, 2018).
  9. T. Müller, F. Rousselle, A. Keller, and J. Novák, Neural control variates, ACM Trans. Graph. 39, 243 (2020).
  10. D. Belomestny, A. Goldman, A. Naumov, and S. Samsonov, Theoretical guarantees for neural control variates in MCMC, Math. Comput. Simul. 220, 382 (2024).
  11. C. Stein, A bound for the error in the normal approximation to the distribution of a sum of dependent random variables, in Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (University of California Press, Berkeley, 1972), Vol 2, pp. 583–602.
  12. S. Si, C. J. Oates, A. B. Duncan, L. Carin, and F.-X. Briol, Scalable control variates for Monte Carlo methods via stochastic optimization, in Monte Carlo and Quasi-Monte Carlo Methods, edited by A. Keller (Springer International Publishing, Cham, 2022), pp. 205–221.
  13. S. Lawrence, Schwinger-Dyson control variates for lattice fermions, arXiv:2404.10707.
  14. K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Netw. 2, 359 (1989).
  15. G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Syst. 2, 303 (1989).
  16. L. F. South, C. J. Oates, A. Mira, and C. Drovandi, Regularized zero-variance control variates, Bayesian Anal. 18, 865 (2023).
  17. T. Bhattacharya, S. Lawrence, and J.-S. Yoo, Control variates for lattice field theory, Phys. Rev. D 109, L031505 (2024).
  18. S. Lawrence and Y. Yamauchi, Mitigating a discrete sign problem with extreme learning machines, arXiv:2312.12636.
  19. W. Detmold, G. Kanwar, M. L. Wagman, and N. C. Warrington, Path integral contour deformations for noisy observables, Phys. Rev. D 102, 014514 (2020).
  20. H. Oh, Control variates with neural networks, Proc. Sci. LATTICE2024 (2025) 051 [arXiv:2501.14614].
  21. J. T. Barron, Continuously differentiable exponential linear units, arXiv:1704.07483.
  22. D. Kingma and J. Ba, Adam: A method for stochastic optimization, in International Conference on Learning Representations (ICLR) (San Diega, CA, 2015).
  23. R. Pascanu, T. Mikolov, and Y. Bengio, On the difficulty of training recurrent neural networks, in Proceedings of the 30th International Conference on International Conference on Machine Learning, ICML’13 (JMLR.org, Cambridge, MA, USA, 2013), Vol. 28, pp. III.1310–III.1318.
  24. S. Lawrence, H. Oh, and Y. Yamauchi, Lattice scalar field theory at complex coupling, Phys. Rev. D 106, 114503 (2022).
  25. G. Catumba and A. Ramos, Stochastic automatic differentiation and the signal to noise problem, arXiv:2502.15570.
  26. A. Alexandru, G. Basar, P. F. Bedaque, and N. C. Warrington, Complex paths around the sign problem, Rev. Mod. Phys. 94, 015006 (2022).
  27. G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602 (2017).
  28. J. Liu, Y. Qi, Z. Y. Meng, and L. Fu, Self-learning Monte Carlo method, Phys. Rev. B 95, 041101 (2017).
  29. D. Levy, M. D. Hoffman, and J. Sohl-Dickstein, Generalizing Hamiltonian Monte Carlo with neural networks, in International Conference on Learning Representations (Amherst, Massachusetts, USA, 2018).
  30. T. Müller, B. Mcwilliams, F. Rousselle, M. Gross, and J. Novák, Neural importance sampling, ACM Trans. Graph. 38, 145 (2019).
  31. N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys. 21, 1087 (1953).
  32. W. K. Hastings, Monte Carlo sampling methods using Markov chains and their applications, Biometrika 57, 97 (1970).
  33. S. Geman and D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell. 6, 721 (1984).
  34. M. H. Quenouille, Approximate tests of correlation in time-series, J. R. Stat. Soc. Ser. B 11, 68 (1949).
  35. M. H. Quenouille, Notes on bias in estimation, Biometrika 43, 353 (1956).

Outline

Information

Phys. Rev. D 112, 074501– Published 7 October, 2025
  • Received 28 May 2025
  • Accepted 8 August 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation