- Physical Review Journals
All JournalsPhysics Magazine

Physical Review E

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Diffusion model-based ensemble generation for highly nonlinear systems with limited ensemble size

Seoleun Shin*

  • *Contact author: seoleuns@kriss.re.kr
PDF

Phys. Rev. E 114, 024218 – Published 21 August, 2026

DOI: https://doi.org/10.1103/xq7z-9cg5

Abstract

We propose a computational strategy that incorporates time-lagged sampling into the reverse process of diffusion-based machine learning models. This approach aims to maintain ensemble diversity and prevent ensemble collapse related to underestimation of uncertainties, particularly in situations with limited ensemble sizes. We demonstrate the method's effectiveness using the Lorenz '96 system as a theoretical testbed, focusing on its highly nonlinear regime without explicitly resolving multiscale interactions. Our approach facilitates stable forecast-analysis cycles at small ensemble sizes without requiring covariance inflation or localization by enhancing the ensemble representation of uncertainties inherent in diffusion models and stochastic dynamics. The proposed models maintain superior temporal coherence. While the probability density function is narrowed under severely undersampled ensembles, stochastic variants recover distributional accuracy as ensemble size increases, whereas deterministic variants show persistent limitations but provide substantial computational efficiency. The method generalizes successfully to systems with increased multiscale complexity and maintains stable long-term performance without drift. Results indicate that active diversity preservation in ensemble generation can enable robust data assimilation in undersampled situations, with potential applications in geophysical flows, plasma systems, and turbulence modeling.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (34)

  1. R. Lam, A. Sanchez-Gonzalez, M. Willson, P. Wirnsberger, M. Fortunato, F. Alet, S. Ravuri, T. Ewalds, Z. Eaton-Rosen, W. Hu, et al., Learning skillful medium-range global weather forecasting, Science 382, 1416 (2023).
  2. I. Price, A. Sanchez-Gonzalez, F. Alet, T. R. Andersson, A. El-Kadi, D. Masters, T. Ewalds, J. Stott, S. Mohamed, P. Battaglia, et al., Probabilistic weather forecasting with machine learning, Nature (London) 637, 84 (2025).
  3. A. J. Charlton-Perez, H. F. Dacre, S. Driscoll, S. L. Gray, B. Harvey, N. J. Harvey, K. M. Hunt, R. W. Lee, R. Swaminathan, R. Vandaele, et al., Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán, npj Clim. Atmos. Sci. 7, 93 (2024).
  4. O. C. Pasche, J. Wider, Z. Zhang, J. Zscheischler, and S. Engelke, Validating deep learning weather forecast models on recent high-impact extreme events, Artificial Intel. Earth Syst. 4, e240033 (2025).
  5. S. Materia, L. P. García, C. van Straaten, S. O, A. Mamalakis, L. Cavicchia, D. Coumou, P. de Luca, M. Kretschmer, and M. Donat, Artificial intelligence for climate prediction of extremes: State of the art, challenges, and future perspectives, WIREs Climate Change 15, e914 (2024).
  6. G. Camps-Valls, M.-Á. Fernández-Torres, K.-H. Cohrs, A. Höhl, A. Castelletti, A. Pacal, C. Robin, F. Martinuzzi, I. Papoutsis, I. Prapas, et al., Artificial intelligence for modeling and understanding extreme weather and climate events, Nature Commun. 16, 1919 (2025).
  7. D. Kochkov, J. Yuval, I. Langmore, P. Norgaard, J. Smith, G. Mooers, M. Klöwer, J. Lottes, S. Rasp, P. Düben, et al., Neural general circulation models for weather and climate, Nature (London) 632, 1060 (2024).
  8. G.-H. Crystalng, D. Mclaughlin, D. Entekhabi, and A. Ahanin, The role of model dynamics in ensemble Kalman filter performance for chaotic systems, Tellus A: Dyn. Meteorol. Oceanogr. 63, 958 (2011).
  9. E. N. Lorenz, in Proceedings Seminar on Predictability (ECMWF, 1996), Vol. 1.
  10. The Lorenz-96 and its GCM analog, 2026, https://m2lines.github.io/L96_demo/notebooks/gcm-analogue.html.
  11. P. D. Dueben and P. Bauer, Challenges and design choices for global weather and climate models based on machine learning, Geosci. Model Dev. 11, 3999 (2018).
  12. S. Hatfield, Evaluating neural network-based surrogates for the forecast model in an ensemble Kalman filter, https://github.com/samhatfield/ml_da, 2019, repository no longer accessible
  13. S. Shin, The predictive skill of neural network models for the large-scale dynamics of the multi-level Lorenz '96 systems, ESS Open Archive (2021).
  14. E. Calvello, S. Reich, and A. M. Stuart, Ensemble Kalman methods: A mean-field perspective, Acta Numer. 34, 123 (2025).
  15. L. Li, R. Carver, I. Lopez-Gomez, F. Sha, and J. Anderson, Generative emulation of weather forecast ensembles with diffusion models, Sci. Adv. 10, eadk4489 (2024).
  16. S. Shin, I.-H. Lee, B. C. Park, and J.-H. Song, Applications of deep learning-based denoising methodologies for scanning electron microscope images, Meas. Sci. Technol. 36, 015406 (2025).
  17. C. Meijer and L. Y. Chen, The rise of diffusion models in time-series forecasting, arXiv:2401.03006.
  18. H. Wen, Y. Lin, Y. Xia, H. Wan, Q. Wen, R. Zimmermann, and Y. Liang, DiffSTG Probabilistic spatio-temporal graph forecasting with denoising diffusion models, in Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems (ACM Press, New York, NY, 2023), pp. 1–12.
  19. M. Li, T. Qu, R. Yao, W. Sun, and M.-F. Moens, Alleviating exposure bias in diffusion models through sampling with shifted time steps, in International Conference on Learning Representations (2024).
  20. R. N. Hoffman and E. Kalnay, Lagged average forecasting, an alternative to Monte Carlo forecasting, Tellus A: Dyn. Meteorol. Oceanogr. 35, 100 (1983).
  21. C. Lu, H. Yuan, B. E. Schwartz, and S. G. Benjamin, Short-range numerical weather prediction using time-lagged ensembles, Weather and forecasting 22, 580 (2007).
  22. C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machine learning (Springer, Berlin, 2006), Vol. 4.
  23. J. Song, C. Meng, and S. Ermon, Denoising diffusion implicit models, in International Conference on Learning Representations (2021).
  24. J. Ho, A. Jain, and P. Abbeel, Denoising diffusion probabilistic models, Adv. Neural Info. Proc. Syst. 33, 6840 (2020).
  25. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, Adv. Neural Info. Proc. Syst. 30, 5998 (2017).
  26. A. Sabour, S. Fidler, and K. Kreis, Align your steps: Optimizing sampling schedules in diffusion models, in Proceedings of the 41st International Conference on Machine Learning (2024).
  27. A. E. Sterk and D. L. van Kekem, Predictability of extreme waves in the Lorenz-96 model near intermittency and quasi-periodicity, Complexity 2017, 1 (2017).
  28. E. Ott, B. R. Hunt, I. Szunyogh, A. V. Zimin, E. J. Kostelich, M. Corazza, E. Kalnay, D. Patil, and J. A. Yorke, A local ensemble Kalman filter for atmospheric data assimilation, Tellus A: Dyn. Meteorol. Oceanogr. 56, 415 (2004).
  29. S. Prabhakaran, ARIMA Model Complete Guide to Time Series Forecasting in Python, Machine Learning Plus (2021), https://www.machinelearningplus.com/time-series/arima-model-time-series-forecasting-python/.
  30. T. Thornes, P. Düben, and T. Palmer, On the use of scale-dependent precision in Earth system modelling, Q. J. R. Meteorol. Soc. 143, 897 (2017).
  31. A. Chattopadhyay, P. Hassanzadeh, and D. Subramanian, Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network, Nonlin. Proc. Geophys. 27, 373 (2020).
  32. J. S. Whitaker and T. M. Hamill, Ensemble data assimilation without perturbed observations, Mon. Weather Rev. 130, 1913 (2002).
  33. A. Jordan, F. Krüger, and S. Lerch, Evaluating probabilistic forecasts with scoringRules, J. Stat. Software 90, 1 (2019).
  34. https://github.com/seoleuns/MLforLorenz96.

Outline

Information

Phys. Rev. E 114, 024218– Published 21 August, 2026
  • Received 28 July 2025
  • Accepted 23 July 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation