- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Neural networks become centroid classifiers when trained on heavily corrupted inputs

Justin Tahmassebpur1,*, Asadullah Bhuiyan2, Hyejin Kim2, and Omri Lesser2

  • *Contact author: jt577@cornell.edu
PDF

Phys. Rev. Research 8, 043088 – Published 23 October, 2026

DOI: https://doi.org/10.1103/qkrg-wbbj

Abstract

Learning from imperfect data is a central theme in machine learning, connecting practical questions of robustness to fundamental questions of learnability. Here, we examine attribute noise: learning from corrupted inputs while keeping the labels intact, a setting that has received considerably less analytical attention than its label-noise counterpart. We consider two types of corruption models: additive noise and replacement noise. Through experiments with multilayer perceptrons (MLPs) on corrupted classification datasets, we find that neural networks remain robust, maintaining well-above-chance accuracy even when inputs are >90% corrupted—far beyond human recognition. To understand this robustness, we analyze infinite-width networks in the heavy-corruption regime using a mean-field-inspired approach and derive a leading-order decision rule for the classification outcome: The network implements a prototype rule, the nearest-class-mean, assigning each test point to the class whose training-set average it most closely resembles. This leading-order decision rule is universal across a broad range of MLP architectures, holding for any depth, as well as a wide class of activation functions and noise distributions. The same centroid mechanism closely matches finite-width network behavior in our experiments and provides an interpretable and analytically tractable account of why learning can succeed even when individual training examples carry almost no signal.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (41)

  1. D. Angluin and P. Laird, Learning from noisy examples, Mach. Learn. 2, 343 (1988).
  2. M. Kearns, Efficient noise-tolerant learning from statistical queries, J. ACM 45, 983 (1998).
  3. A. Blum, A. Kalai, and H. Wasserman, Noise-tolerant learning, the parity problem, and the statistical query model, J. ACM 50, 506 (2003).
  4. B. Frenay and M. Verleysen, Classification in the presence of label noise: A survey, IEEE Trans. Neural Networks Learn. Syst. 25, 845 (2014).
  5. N. Natarajan, I. S. Dhillon, P. Ravikumar, and A. Tewari, Learning with noisy labels, in Advances in Neural Information Processing Systems 26 (NeurIPS 2013) (Curran Associates, Inc., Red Hook, NY, USA, 2013), pp. 1196–1204.
  6. D. Rolnick, A. Veit, S. Belongie, and N. Shavit, Deep learning is robust to massive label noise, arXiv:1705.10694.
  7. C. Northcutt, L. Jiang, and I. Chuang, Confident learning: Estimating uncertainty in dataset labels, J. Artif. Int. Res. 70, 1373 (2021).
  8. T. Emmanuel, T. Maupong, D. Mpoeleng, T. Semong, B. Mphago, and O. Tabona, A survey on missing data in machine learning, J. Big Data 8, 140 (2021).
  9. K. Matsuoka, Noise injection into inputs in back-propagation learning, IEEE Trans. Syst. Man, Cybern. 22, 436 (1992).
  10. L. Holmstrom and P. Koistinen, Using additive noise in back-propagation training, IEEE Trans. Neural Networks 3, 24 (1992).
  11. C. M. Bishop, Training with noise is equivalent to Tikhonov regularization, Neural Comput. 7, 108 (1995).
  12. G. An, The effects of adding noise during backpropagation training on a generalization performance, Neural Comput. 8, 643 (1996).
  13. J. Sietsma and R. J. Dow, Creating artificial neural networks that generalize, Neural Networks 4, 67 (1991).
  14. R. Reed, S. Oh, and R. Marks, Regularization using jittered training data, in [Proceedings 1992] IJCNN International Joint Conference on Neural Networks (IEEE, Piscataway, NJ, USA, 1992), Vol. 3, pp. 147–152.
  15. R. M. Zur, Y. Jiang, L. L. Pesce, and K. Drukker, Noise injection for training artificial neural networks: A comparison with weight decay and early stopping, Med. Phys. 36, 4810 (2009).
  16. O. H. Ramírez-Agudelo Sr, N. Gorea, A. Reif, L. Bonasera, and M. Karl Sr, The role of noisy data in improving CNN robustness for image classification, in Applications of Machine Learning 2025 (SPIE, Bellingham, WA, USA, 2025), Vol. 13606, pp. 197–212.
  17. A. Globerson and S. Roweis, Nightmare at test time: Robust learning by feature deletion, in Proceedings of the 23rd International Conference on Machine Learning, ICML’06 (Association for Computing Machinery, New York, NY, USA, 2006), pp. 353–360.
  18. O. Dekel and O. Shamir, Learning to classify with missing and corrupted features, in Proceedings of the 25th International Conference on Machine Learning, ICML’08 (Association for Computing Machinery, New York, NY, USA, 2008), pp. 216–223.
  19. L. Van Der Maaten, M. Chen, S. Tyree, and K. Q. Weinberger, Learning with marginalized corrupted features, in Proceedings of the 30th International Conference on International Conference on Machine Learning, ICML’13 (JMLR.org, Cambridge, MA, USA, 2013), Vol. 28, pp. I-410–I-418.
  20. D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lakshminarayanan, AugMix: A simple data processing method to improve robustness and uncertainty, in Proceedings of the 8th International Conference on Learning Representations (ICLR, 2020) (OpenReview.net, 2020).
  21. D. Hendrycks and T. G. Dietterich, Benchmarking neural network robustness to common corruptions and perturbations, International Conference on Learning Representations (ICLR) (2019).
  22. N. Mu and J. Gilmer, MNIST-C: A robustness benchmark for computer vision, arXiv:1906.02337.
  23. Y. LeCun, C. Cortes, and C. Burges, MNIST handwritten digit database, ATT Labs [Online], 2010, http://yann.lecun.com/exdb/mnist.
  24. T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha, Deep learning for classical japanese literature, in NeurIPS 2018 Workshop on Machine Learning for Creativity and Design (Montréal, Canada, 2018).
  25. H. Xiao, K. Rasul, and R. Vollgraf, Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms, arXiv:1708.07747.
  26. A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in Advances in Neural Information Processing Systems 31 (NeurIPS 2018) (Curran Associates, Inc., Red Hook, NY, USA, 2018), Vol. 31, pp. 8571–8580.
  27. R. Tibshirani, T. Hastie, B. Narasimhan, and G. Chu, Diagnosis of multiple cancer types by shrunken centroids of gene expression, Proc. Natl. Acad. Sci. USA 99, 6567 (2002).
  28. J. Snell, K. Swersky, and R. S. Zemel, Prototypical networks for few-shot learning, in Advances in Neural Information Processing Systems 30 (NeurIPS 2017) (2017), pp. 4077–4087.
  29. S. Guerriero, B. Caputo, and T. Mensink, Deep nearest class mean classifiers, in ICLR Workshop (2018).
  30. V. Papyan, X. Y. Han, and D. L. Donoho, Prevalence of neural collapse during the terminal phase of deep learning training, Proc. Natl. Acad. Sci. USA 117, 24652 (2020).
  31. M. E. A. Seddik and M. Tamaazousti, Neural networks classify through the class-wise means of their representations, Proc. AAAI Conf. Artif. Intell. 36, 8204 (2022).
  32. V. Kothapalli, Neural collapse: A review on modelling principles and generalization, Transactions on Machine Learning Research (2023).
  33. S. Mei and A. Montanari, The generalization error of random features regression: Precise asymptotics and the double descent curve, Commun. Pure Appl. Math. 75, 667 (2022).
  34. S. Goldt, B. Loureiro, G. Reeves, F. Krzakala, M. Mézard, and L. Zdeborová, The Gaussian equivalence of generative models for learning with shallow neural networks, in Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference (PMLR, Cambridge, MA, USA, 2022), Vol. 145, pp. 426–471.
  35. H. Hu and Y. M. Lu, Universality laws for high-dimensional learning with random features, IEEE Trans. Inf. Theory 69, 1932 (2023).
  36. A. Montanari and B. N. Saeed, Universality of empirical risk minimization, in Proceedings of the 35th Conference on Learning Theory (COLT) (PMLR, Cambridge, MA, USA, 2022), Vol. 178, pp. 4310–4312.
  37. Y. Bahri, B. Hanin, A. Brossollet, V. Erba, C. Keup, R. Pacelli, and J. B. Simon, Les Houches lectures on deep learning at large and infinite width, J. Stat. Mech. (2024) 104012.
  38. D. A. Roberts, S. Yaida, and B. Hanin, The Principles of Deep Learning Theory (Cambridge University Press, Cambridge, MA, USA, 2022), Vol. 46.
  39. R. Price, A useful theorem for nonlinear devices having Gaussian inputs, IRE Trans. Inf. Theory 4, 69 (1958).
  40. M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions (Dover, New York, 1965).
  41. F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain (eds.), NIST Digital Library of Mathematical Functions, Chap. 18, Release 1.2.8 (September 15, 2026), https://dlmf.nist.gov/.

Outline

Information

Phys. Rev. Research 8, 043088– Published 23 October, 2026
  • Received 15 June 2026
  • Accepted 18 September 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation