- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

How to pick the best anomaly detector?

Marie Hein1,*, Gregor Kasieczka2,†, Michael Krämer1,‡, Louis Moureaux2,§, Alexander Mück1,∥, and David Shih3,¶

  • *Contact author: marie.hein@rwth-aachen.de
  • †Contact author: gregor.kasieczka@uni-hamburg.de
  • ‡Contact author: mkraemer@physik.rwth-aachen.de
  • §Contact author: louis.moureaux@cern.ch
  • ∥Contact author: mueck@physik.rwth-aachen.de
  • Contact author: shih@physics.rutgers.edu
PDF

Phys. Rev. D 114, 014034 – Published 16 July, 2026

DOI: https://doi.org/10.1103/9kcn-slvt

Abstract

Anomaly detection has the potential to discover new physics in unexplored regions of the data. However, choosing the best anomaly detector for a given dataset in a model-agnostic way is an important challenge which has hitherto largely been neglected. In this paper, we introduce the data-driven ARGOS metric, which has a sound theoretical foundation and is empirically shown to robustly select the most sensitive anomaly detection model given the data. Focusing on weakly supervised, classifier-based anomaly detection methods, we show that the ARGOS metric outperforms other model selection metrics previously used in the literature, in particular the binary cross-entropy loss. We explore several realistic applications, including hyperparameter tuning as well as architecture and feature selection. In all cases, we demonstrate that ARGOS is highly sensitive to the small amounts of signal present in anomaly detection analyses while at the same time being robust to background overfitting and background template mismodeling.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (54)

  1. G. Kasieczka et al., Rep. Prog. Phys. 84, 124201 (2021).
  2. T. Aarrestad et al., SciPost Phys. 12, 043 (2021).
  3. V. Belis, P. Odagiu, and T. K. Aarrestad, Rev. Phys. 12, 100091 (2023).
  4. HEP ML Community, A living review of machine learning for particle physics, https://iml-wg.github.io/HEPML-LivingReview/.
  5. CMS Collaboration, Rep. Prog. Phys. 88, 067802 (2024).
  6. ATLAS Collaboration, Phys. Rev. Lett. 125, 131801 (2020).
  7. ATLAS Collaboration, Phys. Rev. D 108, 052009 (2023).
  8. ATLAS Collaboration, Phys. Rev. Lett. 132, 081801 (2024).
  9. G. Aad et al. (ATLAS Collaboration), Phys. Rev. D 112, 072009 (2025).
  10. ATLAS Collaboration, Phys. Rev. D 112, 012021 (2025).
  11. CMS Collaboration, Machine-learning techniques for model-independent searches in dijet final states, Technical Report, CERN, CMS Physics Analysis Summary CMS-PAS-MLG-23-002, 2025.
  12. CMS Collaboration, Eur. Phys. J. C 86, 137 (2026).
  13. J. H. Collins, K. Howe, and B. Nachman, Phys. Rev. Lett. 121, 241803 (2018).
  14. J. H. Collins, K. Howe, and B. Nachman, Phys. Rev. D 99, 014038 (2019).
  15. O. Amram and C. M. Suarez, J. High Energy Phys. 01 (2021) 153.
  16. A. Hallin, J. Isaacson, G. Kasieczka, C. Krause, B. Nachman, T. Quadfasel, M. Schlaffer, D. Shih, and M. Sommerhalder, Phys. Rev. D 106, 055006 (2021).
  17. G. Grosso and M. Letizia, Eur. Phys. J. C 85, 4 (2024).
  18. R. T. D’Agnolo and A. Wulzer, Phys. Rev. D 99, 015014 (2019).
  19. R. Das, T. Finke, M. Hein, G. Kasieczka, M. Krämer, A. Mück, and D. Shih, Phys. Rev. D 111, 094041 (2025).
  20. G. Grosso, D. Sengupta, T. Golling, and P. Harris, Eur. Phys. J. C 85, 1074 (2025).
  21. E. M. Metodiev, B. Nachman, and J. Thaler, J. High Energy Phys. 10 (2017) 174.
  22. G. Kasieczka, B. Nachman, and D. Shih, R&D dataset for lhc olympics 2020 anomaly detection challenge, https://zenodo.org/record/6466204 (2019).
  23. D. Shih, Additional QCD background events for LHCO2020 R&D (signal region only), https://zenodo.org/record/5759086 (2021).
  24. T. Sjöstrand, S. Mrenna, and P. Z. Skands, J. High Energy Phys. 05 (2006) 026.
  25. T. Sjöstrand, S. Mrenna, and P. Z. Skands, Comput. Phys. Commun. 178, 852 (2008).
  26. J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaitre, A. Mertens, and M. Selvaggi (DELPHES 3 Collaboration), J. High Energy Phys. 02 (2014) 057.
  27. M. Cacciari, G. P. Salam, and G. Soyez, Eur. Phys. J. C 72, 1896 (2012).
  28. M. Cacciari and G. P. Salam, Phys. Lett. B 641, 57 (2006).
  29. J. Thaler and K. Van Tilburg, J. High Energy Phys. 03 (2011) 015.
  30. J. Thaler and K. Van Tilburg, J. High Energy Phys. 02 (2012) 093.
  31. T. Finke, M. Hein, G. Kasieczka, M. Krämer, A. Mück, P. Prangchaikul, T. Quadfasel, D. Shih, and M. Sommerhalder, Phys. Rev. D 109, 034033 (2023).
  32. B. Nachman and D. Shih, Phys. Rev. D 101, 075042 (2020).
  33. A. Andreassen, B. Nachman, and D. Shih, Phys. Rev. D 101, 095004 (2020).
  34. K. Benkendorfer, L. L. Pottier, and B. Nachman, Phys. Rev. D 104, 035003 (2020).
  35. J. A. Raine, S. Klein, D. Sengupta, and T. Golling, Front. Big Data 6, 899345 (2022).
  36. A. Hallin, G. Kasieczka, T. Quadfasel, D. Shih, and M. Sommerhalder, Phys. Rev. D 107, 114012 (2022).
  37. T. Golling, S. Klein, R. Mastandrea, and B. Nachman, Phys. Rev. D 107, 096025 (2023).
  38. T. Golling, G. Kasieczka, C. Krause, R. Mastandrea, B. Nachman, J. A. Raine, D. Sengupta, D. Shih, and M. Sommerhalder, Eur. Phys. J. C 84, 241 (2023).
  39. R. Das and D. Shih, Phys. Rev. D 112, 074040 (2025).
  40. M. Leigh, D. Sengupta, B. Nachman, and T. Golling, J. High Energy Phys. 12 (2025) 105.
  41. I. Oleksiyuk, S. Voloshynovskiy, and T. Golling, J. High Energy Phys. 07 (2025) 177.
  42. F. Pedregosa et al., J. Mach. Learn. Res. 12, 2825 (2011).
  43. R. Das, G. Kasieczka, and D. Shih, arXiv:2312.11629.
  44. R. Das, M. Hein, G. Kasieczka, M. Krämer, L. Lang, R. Mastandrea, L. Moureaux, A. Mück, and D. Shih, arXiv:2604.20965.
  45. M. Hein, ARGOS metric for anomaly detection, https://github.com/mariehein/ARGOS-metric-for-anomaly-detection (2025).
  46. R. Das, Density Estimation for “Accurate and robust methods for direct background estimation in resonant anomaly detection, https://github.com/rd804/cut_and_count_FM (2024).
  47. K. He, X. Zhang, S. Ren, and J. Sun, arXiv:1512.03385.
  48. C. Durkan, A. Bekasov, I. Murray, and G. Papamakarios, nflows: Normalizing flows in pytorch (2020), 10.5281/zenodo.4296287.
  49. J. Ansel et al., in 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS ’24) (ACM, New York, NY, USA, 2024), 10.1145/3620665.3640366.
  50. O. Amram, J. Birk, and M. Sommerhalder, sk_cathode, https://github.com/uhh-pd-ml/sk_cathode (2024).
  51. D. P. Kingma and J. Ba, arXiv:1412.6980.
  52. M. Freytsis, M. Perelstein, and Y. C. San, J. High Energy Phys. 02 (2023) 220.
  53. M. Hein, B. Nachman, and D. Shih, arXiv:2512.13787.
  54. R. Gambhir, R. Mastandrea, B. Nachman, and J. Thaler, Phys. Rev. Lett. 135, 021902 (2025).

Outline

Information

Phys. Rev. D 114, 014034– Published 16 July, 2026
  • Received 26 January 2026
  • Accepted 29 June 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation