- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Foundation model framework for all tasks involving jet physics

Wahid Bhimji1, Chris Harris1, Vinicius Mikuni2,*, and Benjamin Nachman3,4,†

  • *Contact author: vmikuni@hepl.phys.nagoya-u.ac.jp
  • †Contact author: nachman@stanford.edu
PDF

Phys. Rev. D 113, 032020 – Published 26 February, 2026

DOI: https://doi.org/10.1103/knmd-f5jm

Abstract

Foundation models use large datasets to build an effective representation of data that can be deployed on diverse downstream tasks. Previous research developed the omnilearn foundation model for jet physics, using unique properties of particle physics, and showed that it could significantly advance discovery potential across collider experiments. This paper introduces a major upgrade, resulting in the omnilearned framework. This framework has three new elements: (1) updates to the model architecture and training, (2) using over 1×109 jets used for training, and (3) providing well-documented software for accessing all datasets and models. We demonstrate omnilearned with three representative tasks: top-quark jet tagging with the community delphes-based benchmark dataset, b tagging with ATLAS full simulation, and anomaly detection with CMS experimental data. In each case, omnilearned is the state of the art, further expanding the discovery potential of past, current, and future collider experiments.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (99)

  1. R. Kogler et al., Rev. Mod. Phys. 91, 045003 (2019).
  2. A. J. Larkoski, I. Moult, and B. Nachman, Phys. Rep. 841, 1 (2020).
  3. S. Marzani, G. Soyez, and M. Spannowsky, Looking Inside Jets: An Introduction to Jet Substructure and Boosted-Object Phenomenology, Vol. 958 (Springer, New York, 2019).
  4. M. Feickert and B. Nachman, arXiv:2102.02770.
  5. J. Birk, A. Hallin, and G. Kasieczka, Mach. Learn. Sci. Technol. 5, 035031 (2024).
  6. A. Hallin, in 2nd European AI for Fundamental Physics Conference (2025).
  7. P. Harris, M. Kagan, J. Krupa, B. Maier, and N. Woodward, Phys. Rev. D 111, 032010 (2025).
  8. T. Golling, L. Heinrich, M. Kagan, S. Klein, M. Leigh, M. Osadchy, and J. A. Raine, Mach. Learn. Sci. Tech. 5, 035074 (2024).
  9. M. Leigh, S. Klein, F. Charton, T. Golling, L. Heinrich, M. Kagan, I. Ochoa, and M. Osadchy, arXiv:2409.12589.
  10. J. Bardhan, R. Agrawal, A. Tilak, C. Neeraj, and S. Mitra, arXiv:2502.03933.
  11. V. Mikuni and B. Nachman, Phys. Rev. D 111, 054015 (2025).
  12. V. Mikuni and B. Nachman, Phys. Rev. D 111, L051504 (2025).
  13. J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens, and M. Selvaggi (DELPHES 3 Collaboration), J. High Energy Phys. 02 (2014) 057.
  14. H. Qu, C. Li, and S. Qian, arXiv:2202.03772.
  15. S. Agostinelli et al. (GEANT4 Collaboration), Nucl. Instrum. Methods Phys. Res., Sect. A 506, 250 (2003).
  16. P. T. Komiske, E. M. Metodiev, and J. Thaler, J. High Energy Phys. 01 (2019) 121.
  17. ATLAS Collaboration, Deep Sets based Neural Networks for Impact Parameter Flavour Tagging in ATLAS, Tech. Rep. ATL-PHYS-PUB-2020-014, CERN, Geneva, 2020.
  18. E. A. Moreno, O. Cerri, J. M. Duarte, H. B. Newman, T. Q. Nguyen, A. Periwal, M. Pierini, A. Serikova, M. Spiropulu, and J.-R. Vlimant, Eur. Phys. J. C 80, 58 (2020).
  19. H. Qu and L. Gouskos, Phys. Rev. D 101, 056019 (2020).
  20. C. Shimmin, arXiv:2107.02908.
  21. V. Mikuni and F. Canelli, Eur. Phys. J. Plus 135, 463 (2020).
  22. V. Mikuni and F. Canelli, Mach. Learn. Sci. Technol. 2, 035027 (2021).
  23. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, J. Mach. Learn. Res. 15, 1929 (2014), https://jmlr.org/papers/v15/srivastava14a.html.
  24. S. Goriely, A. Choplin, W. Ryssens, and I. Kullmann, in Proceedings of the 28th International Nuclear Physics Conference, Vol. 2586 (AIP Publishing, Melville, NY, USA, 2022), p. 012104.
  25. M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, in Advances in Neural Information Processing Systems, Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Curran Associates, Inc., Red Hook, NY, USA, 2020), pp. 7537–7547.
  26. Y. Wu, K. Wang, C. Li, H. Qu, and J. Zhu, Chin. Phys. C 49, 013110 (2025).
  27. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, arXiv:1706.03762.
  28. J. Zhu, X. Chen, K. He, Y. LeCun, and Z. Liu, in Proceedings of the Computer Vision and Pattern Recognition Conference (IEEE, Piscataway, NJ, USA, 2025), pp. 14901–14911.
  29. T. Salimans and J. Ho, in International Conference on Learning Representations (OpenReview (ICLR proceedings are published electronically), Amherst, MA, USA, 2022).
  30. V. Mikuni, B. Nachman, and M. Pettee, Phys. Rev. D 108, 036025 (2023).
  31. J. Y. Araz, V. Mikuni, F. Ringer, N. Sato, F. T. Acosta, and R. Whitehill, Phys. Lett. B 868, 139694 (2025).
  32. Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, arXiv:2210.02747.
  33. C. Li et al., arXiv:2405.12972.
  34. O. Amram, L. Anzalone, J. Birk, D. A. Faroughy, A. Hallin, G. Kasieczka, M. Krämer, I. Pang, H. Reyes-Gonzalez, and D. Shih, Mach. Learn. Sci. Technol. 6, 030601 (2025).
  35. G. Aad et al. (ATLAS Collaboration), J. Instrum. 19, P08018 (2024).
  36. J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, and M. Zaro, J. High Energy Phys. 07 (2014) 079.
  37. T. Sjöstrand, S. Mrenna, and P. Z. Skands, J. High Energy Phys. 05 (2006) 026.
  38. T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands, Comput. Phys. Commun. 191, 159 (2015).
  39. A. Mertens, J. Phys. Conf. Ser. 608, 012045 (2015).
  40. M. Selvaggi, J. Phys. Conf. Ser. 523, 012033 (2014).
  41. M. Cacciari and G. P. Salam, Phys. Lett. B 641, 57 (2006).
  42. M. Cacciari, G. P. Salam, and G. Soyez, Eur. Phys. J. C 72, 1896 (2012).
  43. M. Cacciari, G. P. Salam, and G. Soyez, J. High Energy Phys. 04 (2008) 063.
  44. G. Aad et al. (ATLAS Collaboration), Eur. Phys. J. C 70, 823 (2010).
  45. R. D. Ball et al., Nucl. Phys. B867, 244 (2013).
  46. A. Buckley, in Proceedings of the 6th International Workshop on Multiple Partonic Interactions at the LHC (World Scientific Publishing, Singapore, 2014), p. 29.
  47. G. Aad et al. (ATLAS Collaboration), Eur. Phys. J. C 81, 334 (2021).
  48. A. J. Larkoski, S. Marzani, G. Soyez, and J. Thaler, J. High Energy Phys. 05 (2014) 146.
  49. P. Berta, M. Spousta, D. W. Miller, and R. Leitner, J. High Energy Phys. 06 (2014) 092.
  50. P. Berta, L. Masetti, D. W. Miller, and M. Spousta, J. High Energy Phys. 08 (2019) 175.
  51. M. Cacciari, G. P. Salam, and G. Soyez, Eur. Phys. J. C 75, 59 (2015).
  52. H. Jung, Comput. Phys. Commun. 86, 147 (1995).
  53. K. Charchula, G. A. Schuler, and H. Spiesberger, Comput. Phys. Commun. 81, 381 (1994).
  54. R. Brun, F. Bruyant, M. Maire, A. C. McPherson, and P. Zanarini, CERN, (1987).
  55. D. Britzger, S. Levonian, S. Schmitt, and D. South (H1 Collaboration), EPJ Web Conf. 251, 03004 (2021).
  56. M. Peez, Search for deviations from the standard model in high transverse energy processes at the electron proton collider HERA, Other thesis, Université de Lyon, 2003.
  57. S. Hellwig, Untersuchung der D*−πslow Double Tagging Methode in Charmanalysen, Master’s thesis, Universität Hamburg, 2004.
  58. B. Portheault, First measurement of charged and neutral current cross sections with the polarized positron beam at HERA II and QCD-electroweak analyses, Other thesis, Université Paris-Sud, 2005.
  59. “Perlmutter system”, https://docs.nersc.gov/systems/perlmutter/system_details/, (accessed: 2022-05-04).
  60. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., PyTorch: An imperative style, high-performance deep learning library, in Advances in Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, USA, 2019).
  61. I. Loshchilov and F. Hutter, arXiv:1608.03983.
  62. X. Chen, C. Liang, D. Huang, E. Real, K. Wang, H. Pham, X. Dong, T. Luong, C.-J. Hsieh, Y. Lu et al., Advances in Neural Information Processing Systems (Curran Associates, Inc., Red Hook, NY, USA, 2024).
  63. A. Butter et al., SciPost Phys. 7, 014 (2019).
  64. A. Bogatskiy, B. Anderson, J. T. Offermann, M. Roussi, D. W. Miller, and R. Kondor, arXiv:2006.04780.
  65. S. Gong, Q. Meng, J. Zhang, H. Qu, C. Li, S. Qian, W. Du, Z.-M. Ma, and T.-Y. Liu, J. High Energy Phys. 07 (2022) 030.
  66. A. Bogatskiy, T. Hoffman, D. W. Miller, and J. T. Offermann, arXiv:2211.00454.
  67. A. Hammad and M. Nojiri, J. High Energy Phys. 06 (2024) 176.
  68. J. Brehmer, V. Bresó, P. de Haan, T. Plehn, H. Qu, J. Spinner, and J. Thaler, SciPost Phys. 19, 108 (2025).
  69. G. Aad et al. (ATLAS Collaboration), Nat. Commun. 17, 541 (2026).
  70. Atlas Collaboration, “Atlas tt¯ simulation for ml-based jet flavour tagging (jetset)”, CERN Open Data Portal, (2025) 10.7483/OPENDATA.ATLAS.QG8W.TO8P.
  71. S. Alioli, P. Nason, C. Oleari, and E. Re, J. High Energy Phys. 06 (2010) 043.
  72. S. Frixione, P. Nason, and C. Oleari, J. High Energy Phys. 11 (2007) 070.
  73. P. Nason, J. High Energy Phys. 11 (2004) 040.
  74. S. Frixione, P. Nason, and G. Ridolfi, J. High Energy Phys. 09 (2007) 126.
  75. E. M. Metodiev, B. Nachman, and J. Thaler, J. High Energy Phys. 10 (2017) 174.
  76. J. H. Collins, K. Howe, and B. Nachman, Phys. Rev. Lett. 121, 241803 (2018).
  77. J. H. Collins, K. Howe, and B. Nachman, Phys. Rev. D 99, 014038 (2019).
  78. G. Kasieczka et al., Rep. Prog. Phys. 84, 124201 (2021).
  79. T. Aarrestad et al., SciPost Phys. 12, 043 (2021).
  80. G. Karagiorgi, G. Kasieczka, S. Kravitz, B. Nachman, and D. Shih, arXiv:2112.03769.
  81. A. Hallin, J. Isaacson, G. Kasieczka, C. Krause, B. Nachman, T. Quadfasel, M. Schlaffer, D. Shih, and M. Sommerhalder, Phys. Rev. D 106, 055006 (2021).
  82. C. L. Cheng, G. Singh, and B. Nachman, Phys. Rev. Lett. 135, 021801 (2025).
  83. C. L. Cheng, R. Das, R. Li, R. Mastandrea, V. Mikuni, B. Nachman, D. Shih, and G. Singh, arXiv:2506.00119.
  84. G. Aad et al. (ATLAS Collaboration), Phys. Rev. Lett. 125, 131801 (2020).
  85. G. Aad et al. (ATLAS Collaboration), Phys. Rev. D 112, 072009 (2025).
  86. V. Chekhovsky et al. (CMS Collaboration), Rep. Prog. Phys. 88, 067802 (2025).
  87. R. Gambhir, R. Mastandrea, B. Nachman, and J. Thaler, Phys. Rev. Lett. 135, 021902 (2025).
  88. D. Shih, M. R. Buckley, L. Necib, and J. Tamanas, Mon. Not. R. Astron. Soc. 509, 5992 (2021).
  89. D. Shih, M. R. Buckley, and L. Necib, Mon. Not. R. Astron. Soc. 529, 4745 (2024).
  90. M. Pettee, S. Thanvantri, B. Nachman, D. Shih, M. R. Buckley, and J. H. Collins, Mon. Not. R. Astron. Soc. 527, 8459 (2023).
  91. D. Sengupta, S. Mulligan, D. Shih, J. A. Raine, and T. Golling, Mon. Not. R. Astron. Soc. 536, 1104 (2024).
  92. A. Hallin, D. Shih, C. Krause, and M. R. Buckley, arXiv:2509.08064.
  93. O. Knapp, O. Cerri, G. Dissertori, T. Q. Nguyen, M. Pierini, and J.-R. Vlimant, Eur. Phys. J. Plus 136, 236 (2021).
  94. T. Aaltonen et al. (CDF Collaboration), Phys. Rev. D 79, 112002 (2009).
  95. G. Aad et al. (ATLAS Collaboration), Phys. Lett. B 754, 302 (2016).
  96. V. Khachatryan et al. (CMS Collaboration), Phys. Rev. Lett. 117, 031802 (2016).
  97. E. Buhmann, C. Ewen, G. Kasieczka, V. Mikuni, B. Nachman, and D. Shih, Phys. Rev. D 109, 055015 (2024).
  98. B. Nachman and D. Shih, Phys. Rev. D 101, 075042 (2020).
  99. V. Mikuni and C. Harris, Viniciusmikuni/omnilearned: 0.2.0, (2026) 10.5281/zenodo.18489564.

Outline

Information

Phys. Rev. D 113, 032020– Published 26 February, 2026
  • Received 11 December 2025
  • Accepted 4 February 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation