- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Letters

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Lost in Retraining: Closed-Loop Learning and Model Collapse in Exponential Families

Fariba Jangjoo1, Giovanni di Sarra1, Matteo Marsili2, and Yasser Roudi3

PDF

Phys. Rev. Lett. 136, 197301 – Published 14 May, 2026

DOI: https://doi.org/10.1103/156q-3ngc

Abstract

Closed-loop learning is the process of repeatedly estimating a model from data generated from the model itself. It is receiving great attention due to the possibility that large neural network models may, in the future, be primarily trained with data generated by artificial neural networks themselves. We study this process for models that belong to exponential families, deriving equations of motion that govern the dynamics of the parameters. We show that maximum likelihood estimation of the parameters endows sufficient statistics with the martingale property and that as a result the process converges to absorbing states that amplify initial biases present in the data. However, we show that, for exponential families, this outcome may be prevented if, at each closed-loop retraining iteration, the data contains at least one data point generated from a ground truth model, by relying on maximum a posteriori estimation or by introducing regularization.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (39)

  1. S. Alemohammad, J. Casco-Rodriguez, L. Luzi, A. I. Humayun, H. Babaei, D. LeJeune, A. Siahkoohi, and R. Baraniuk, in International Conference on Learning Representations (ICLR) (2024).
  2. Q. Bertrand, A. J. Bose, A. Duplessis, M. Jiralerspong, and G. Gidel, in International Conference on Learning Representations (ICLR) (2024).
  3. M. Marchi, S. Soatto, P. Chaudhari, and P. Tabuada, in Proceedings of the 2024 IEEE 63rd Conference on Decision and Control (CDC) (2024), pp. 1524–1530.
  4. I. Shumailov, Z. Shumaylov, Y. Zhao, N. Papernot, R. Anderson, and Y. Gal, Nature (London) 631, 755 (2024).
  5. Y. Guo, G. Shang, M. Vazirgiannis, and C. Clavel, in Findings of the Association for Computational Linguistics: NAACL 2024, edited by K. Duh, H. Gomez, and S. Bethard (Association for Computational Linguistics, Mexico, 2024), pp. 3589–3604.
  6. M. Briesch, D. Sobania, and F. Rothlauf, arXiv:2311.16822.
  7. E. Dohmatob, Y. Feng, and J. Kempe, Adv. Neural Inf. Process. Syst. 37, 46979 (2024).
  8. H. A. Vu, G. Reeves, and E. Wenger, arXiv:2505.21677.
  9. P. Villalobos, A. Ho, J. Sevilla, T. Besiroglu, L. Heim, and M. Hobbhahn, arXiv:2211.04325.
  10. H. Thanh-Tung and T. Tran, in Proceedings of the 2020 International Joint Conference on Neural Networks (2020), pp. 1–10.
  11. G. Biroli, T. Bonnaire, V. De Bortoli, and M. Mézard, Nat. Commun. 15, 9957 (2024).
  12. A. R. Barron and C.-H. Sheu, Ann. Stat. 19, 1347 (1991).
  13. S.-i. Amari, Information Geometry and Its Applications (Springer, New York, 2016).
  14. G. Casella and R. Berger, Statistical Inference (CRC Press, New York, 2024).
  15. R. Tibshirani, J. R. Stat. Soc. Ser. B 58, 267 (1996).
  16. É. Roldán, I. Neri, R. Chetrite, S. Gupta, S. Pigolotti, F. Jülicher, and K. Sekimoto, Adv. Phys. 72, 1 (2023).
  17. W. Feller, An Introduction to Probability Theory and its Applications (John Wiley & Sons, New York, 1968), Vol. 1.
  18. See Supplemental Material at http://link.aps.org/supplemental/10.1103/156q-3ngc for details of the calculations and any relevant numerical results.
  19. B. Derrida, Phys. Rev. B 24, 2613 (1981).
  20. C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 3rd ed. (Springer, Berlin, Heidelberg, 2009).
  21. J. C. Cox, J. E. Ingersoll, S. A. Ross et al., Econometrica 53, 385 (1985).
  22. N. Zhang, S. Ding, J. Zhang, and Y. Xue, Neurocomputing;Variable Star Bulletin 275, 1186 (2018).
  23. G. di Sarra, B. Bravi, and Y. Roudi, Europhys. Lett. 149, 21002 (2025).
  24. G. Hinton, Rev. Mod. Phys. 97, 030502 (2025).
  25. A. Decelle, C. Furtlehner, and B. Seoane, Adv. Neural Inf. Process. Syst. 34, 5345 (2021).
  26. N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, in International Conference on Machine Learning (2019), pp. 5301–5310.
  27. E. Francazi, A. Lucchi, and M. Baity-Jesi, arXiv:2306.00809.
  28. A. Jain, R. Nobahari, A. Baratin, and S. Sarao Mannelli, Adv. Neural Inf. Process. Syst. 37, 24435 (2024).
  29. S. Watanabe, J. Mach. Learn. Res. 14, 867 (2013).
  30. T. Hayase and R. Karakida, in International Conference on Artificial Intelligence and Statistics (2021), pp. 334–342.
  31. W. Cao, A. Dytso, M. Fauß, H. V. Poor, and G. Feng, in Proceedings of the 2020 IEEE International Symposium on Information Theory (2020), pp. 2216–2221.
  32. C. L. Baker, R. Saxe, and J. B. Tenenbaum, Cognition 113, 329 (2009).
  33. A. Najar, E. Bonnet, B. Bahrami, and S. Palminteri, PLoS Biol. 18, e3001028 (2020).
  34. C. J. Charpentier, K. Iigaya, and J. P. O’Doherty, Neuron 106, 687 (2020).
  35. S. Bikhchandani, D. Hirshleifer, and I. Welch, J. Polit. Econ. 100, 992 (1992).
  36. E. M. Witzel, The Origins of the World’s Mythologies (Oxford University Press, New York, 2012).
  37. F. J. Martini, T. Guillamón-Vivancos, V. Moreno-Juan, M. Valdeolmillos, and G. López-Bendito, Neuron 109, 2519 (2021).
  38. R. O. Wong, Annu. Rev. Neurosci. 22, 29 (1999).
  39. Y. Roudi, Repository for closed loop learning, https://github.com/yasser-roudi/-closed-loop-learning; Accessed: 29 March 2026.

Outline

Information

Phys. Rev. Lett. 136, 197301– Published 14 May, 2026
  • Received 6 April 2025
  • Revised 9 March 2026
  • Accepted 24 March 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation