- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Anomaly-preserving contrastive neural embeddings for end-to-end model-independent searches at the LHC

Kyle Metzger1,*, Lana Xu2, Mia Sodini2, Thea K. Årrestad1,†, Katya Govorkova2,‡, Gaia Grosso2,3,4,§, and Philip Harris2,3,∥

  • *Contact author: kyle.metzger@bluewin.ch
  • †Contact author: thea.aarrestad@cern.ch
  • ‡Contact author: ekaterina.govorkova@cern.ch
  • §Contact author: gaia.grosso@cern.ch
  • ∥Contact author: philip.coleman.harris@cern.ch
PDF

Phys. Rev. D 112, 072011 – Published 23 October, 2025

DOI: https://doi.org/10.1103/5n77-ynsp

Abstract

Anomaly detection—identifying deviations from Standard Model predictions—is a key challenge at the Large Hadron Collider due to the size and complexity of its datasets. This is typically addressed by transforming high-dimensional detector data into lower-dimensional, physically meaningful features. We tackle feature extraction for anomaly detection by learning powerful low-dimensional representations via contrastive neural embeddings. This approach preserves potential anomalies indicative of new physics and enables rare signal extraction using novel machine-learning-based statistical methods for signal-independent hypothesis testing. We compare supervised and self-supervised contrastive learning methods, for both Multilayer perceptron-and transformer-based neural embeddings, trained on the kinematic observables of physics objects in LHC collision events. The learned embeddings serve as input representations for signal-agnostic statistical detection methods in inclusive final states. We achieve significant improvement in discovery power for both rare new physics signals and rare Standard Model processes across diverse final states, demonstrating its applicability for efficiently searching for diverse signals simultaneously. We study the impact of architectural choices, contrastive loss formulations, supervision levels, and embedding dimensionality on anomaly detection performance. We show that the optimal representation for background classification does not always maximize sensitivity to new physics signals, revealing an inherent trade-off between background structure preservation and anomaly enhancement. We demonstrate that combining compression with domain knowledge for label encoding produces the most effective data representation for statistical discovery of anomalies.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (60)

  1. OpenAI, GPT-4 technical report, 2024, arXiv:2303.08774.
  2. Y. Yang, S. Mandt, and L. Theis, An introduction to neural data compression. Found. Trends®, Comput. Graph. Vision 15, 113 (2023).
  3. V. M. Abazov et al., A quasi model independent search for new physics at large transverse momentum, Phys. Rev. D 64, 012004 (2001).
  4. B. Abbott et al., A quasi model independent search for new high pT physics at D0, Phys. Rev. Lett. 86, 3712 (2001).
  5. A. Aktas et al., A general search for new phenomena in ep scattering at HERA, Phys. Lett. B 602, 14 (2004).
  6. T. Aaltonen et al., Model-independent and quasi-model-independent search for new physics at CDF, Phys. Rev. D 78, 012002 (2008).
  7. F. D. Aaron et al., A general search for new phenomena at HERA, Phys. Lett. B 674, 257 (2009).
  8. T. Aaltonen et al., Global search for new physics with 2.0  fb−1 at CDF, Phys. Rev. D 79, 011101 (2009).
  9. V. M. Abazov et al., Model independent search for new phenomena in pp¯ collisions at s=1.96  TeV, Phys. Rev. D 85, 092015 (2012).
  10. , A general search for new phenomena with the ATLAS detector in pp collisions at s=8  TeV, ATLAS Report No. ATLAS-CONF-2014-006, 2014, https://cds.cern.ch/record/1666536.
  11. A. M. Sirunyan et al., MUSiC: A model-unspecific search for new physics in proton–proton collisions at s=13  TeV, Eur. Phys. J. C 81, 629 (2021).
  12. ATLAS Collaboration, Dijet resonance search with weak supervision using s=13  TeV pp collisions in the ATLAS detector, Phys. Rev. Lett. 125, 131801 (2020).
  13. , Model-agnostic search for dijet resonances with anomalous jet substructure in proton-proton collisions at s=13  TeV, Rep. Prog. Phys. 88, 067802 (2025).
  14. G. Aad et al., Anomaly detection search for new resonances decaying into a Higgs boson and a generic new particle X in hadronic final states using s=13  TeV pp collisions with the ATLAS detector, Phys. Rev. D 108, 052009 (2023).
  15. G. Aad et al., Search for new phenomena in two-body invariant mass distributions using unsupervised machine learning for anomaly detection at s=13  TeV with the ATLAS detector. Phys. Rev. Lett. 132, 081801 (2024).
  16. G. Aad et al., Weakly supervised anomaly detection for resonant new physics in the dijet final state using proton-proton collisions at s=13  TeV with the ATLAS detector, arXiv:2502.09770.
  17. V. Belis, P. Odagiu, and T. K. Aarrestad, Machine learning for anomaly detection in particle physics, Rev. Phys. 12, 100091 (2024).
  18. T. Finke et al., Back to the roots: Tree-based algorithms for weakly supervised anomaly detection, Phys. Rev. D 109, 034033 (2024).
  19. M. Freytsis, M. Perelstein, and Y. C. San, Anomaly detection in the presence of irrelevant features, J. High Energy Phys. 02 (2024) 220.
  20. M. Letizia et al., Learning new physics efficiently with nonparametric methods, Eur. Phys. J. C 82, 879 (2022).
  21. G. Grosso, Anomaly-aware summary statistic from data batches, J. High Energy Phys. 12 (2024) 093.
  22. R. T. d’Agnolo, G. Grosso, M. Pierini, A. Wulzer, and M. Zanetti, Learning new physics from an imperfect machine, Eur. Phys. J. C 82, 275 (2022).
  23. A. Khot, X. Wang, A. Roy, V. Kindratenko, and M. S. Neubauer, Evidential deep learning for uncertainty quantification and out-of-distribution detection in jet identification using deep neural networks, Mach. Learn. Sci. Tech. 6, 035003 (2025).
  24. S. E. Park, P. Harris, and B. Ostdiek, Neural embedding: Learning the embedding of the manifold of physics data, J. High Energy Phys. 07 (2023) 108.
  25. P. Harris, M. Kagan, J. Krupa, B. Maier, and N. Woodward, Re-simulation-based self-supervised learning for pre-training foundation models, Phys. Rev. D 111, 032010 (2025).
  26. B. M. Dillon et al., Symmetries, safety, and self-supervision, SciPost Phys. 12, 188 (2022).
  27. V. Mikuni and B. Nachman, OmniLearn: A method to simultaneously facilitate all jet physics tasks, Phys. Rev. D 111, L051504 (2025).
  28. M. Leigh et al., Is tokenization needed for masked particle modelling?, arXiv:2409.12589.
  29. J. Birk, A. Hallin, and G. Kasieczka, OmniJet-α: The first cross-task foundation model for particle physics, Mach. Learn. Sci. Tech. 5, 035031 (2024).
  30. A. J. Wildridge et al., Bumblebee: Foundation model for particle physics discovery, in Proceedings of the 38th Conference on Neural Information Processing Systems (2024), arXiv:2412.07867.
  31. J. Bardhan, R. Agrawal, A. Tilak, C. Neeraj, and S. Mitra, HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture, arXiv:2502.03933.
  32. T. Golling et al., Masked particle modeling on sets: Towards self-supervised high energy physics foundation models, Mach. Learn. Sci. Tech. 5, 035074 (2024).
  33. C. Li et al., Accelerating resonance searches via signature-oriented pre-training, arXiv:2405.12972.
  34. B. M. Dillon, R. Mastandrea, and B. Nachman, Self-supervised anomaly detection for new physics, Phys. Rev. D 106, 056005 (2022).
  35. L. Favaro, M. Krämer, T. Modak, T. Plehn, and J. Rüschkamp, Semi-visible jets, energy-based models, and self-supervision, SciPost Phys. 18, 042 (2025).
  36. B. M. Dillon, L. Favaro, F. Feiden, T. Modak, and T. Plehn, Anomalies, representations, and self-supervision, SciPost Phys. Core 7, 056 (2024).
  37. Y. Bengio, A. Courville, and P. Vincent, Representation learning: A review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798 (2013).
  38. X. Chen and K. He, Exploring simple siamese representation learning, in Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2021), pp. 15745–15753.
  39. P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, Supervised Contrastive Learning, in Advances in Neural Information Processing Systems, Vol. 33 (Curran Associates, Inc., Red Hook, NY, 2020), pp. 18661–18673.
  40. T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, A simple framework for contrastive learning of visual representations, in Proceedings of the 37th International Conference on Machine Learning (PMLR, 2020), Vol. 119, pp. 1597–1607.
  41. A. Bardes, J. Ponce, and Y. LeCun, VICReg: Variance-invariance-covariance regularization for self-supervised learning, arXiv:2105.04906.
  42. G. Grosso, M. Letizia, M. Pierini, and A. Wulzer, Goodness of fit by Neyman-Pearson testing, SciPost Phys. 16, 123 (2024).
  43. R. T. D’Agnolo and A. Wulzer, Learning new physics from a machine, Phys. Rev. D 99, 015014 (2019).
  44. J. Neyman and E. S. Pearson, On the problem of the most efficient tests of statistical hypotheses, Phil. Trans. R. Soc. A 231, 289 (1933).
  45. G. Meanti, L. Carratino, L. Rosasco, and A. Rudi, Kernel methods through the roof: Handling billions of points efficiently, in Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS ’20) (Curran Associates Inc., Red Hook, NY, USA, 2020), pp. 14410–14422.
  46. E. Govorkova et al., LHC physics dataset for unsupervised new physics detection at 40 MHz, Nat. Sci. Data 9, 118 (2022).
  47. T. Sjöstrand et al., An introduction to pythia8.2, Comput. Phys. Commun. 191, 159 (2015).
  48. J. de Favereau et al., delphes3, A modular framework for fast simulation of a generic collider experiment, J. High Energy Phys. 02 (2014) 057.
  49. https://www.drawio.com/.
  50. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, Adv. Neural Inf. Process. Syst. 30, 5998 (2017).
  51. Pytorch transformer, https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html.
  52. M. Caron et al., Self-supervised vision transformers with dino, https://github.com/facebookresearch/dino/tree/main.
  53. Y. You, I. Gitman, and B. Ginsburg, Large batch training of convolutional networks (2017).
  54. I. Loshchilov and F. Hutter, Decoupled weight decay regularization, in Proceedings of the 7th International Conference on Learning Representations (ICLR, New Orleans, LA, 2019).
  55. B. M. Dillon, L. Favaro, F. Feiden, T. Modak, and T. Plehn, Anomalies, representations, and self-supervision, SciPost Phys. 7, 056 (2024).
  56. E. Govorkova et al., Autoencoders on field-programmable gate arrays for real-time, unsupervised new physics detection at 40 mhz at the large hadron collider, Nat. Mach. Intell. 4, 154 (2022).
  57. L. McInnes, J. Healy, N. Saul, and L. Großberger, UMAP: Uniform manifold approximation and projection, J. Open Source Softwaare 3, 861 (2018).
  58. S. E. Park, D. Rankin, S.-M. Udrescu, M. Yunus, and P. Harris, Quasi anomalous knowledge: Searching for new physics with embedded knowledge, J. High Energy Phys. 06 (2021) 030.
  59. 2024 Data Collected with AXOL1TL anomaly detection at the CMS level-1 trigger (2024), https://cds.cern.ch/record/2904695.
  60. R. Ardino et al., A 40 MHz level-1 trigger scouting system for the CMS phase-2 upgrade. Nucl. Instrum. Methods Phys. Res., Sect. A 1047, 167805 (2023).

Outline

Information

Phys. Rev. D 112, 072011– Published 23 October, 2025
  • Received 21 March 2025
  • Accepted 5 September 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation