- Physical Review Journals
All JournalsPhysics Magazine

Physical Review X

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Transformers for Charged Particle Track Reconstruction in High-Energy Physics

Samuel Van Stroud1,*, Philippa Duckett1,†, Max Hart1,‡, Nikita Pond1,§, Sébastien Rettie1,2,∥, Gabriel Facini1,¶, and Tim Scanlon1,**

  • *Contact author: sam.vanstroud.18@ucl.ac.uk
  • †Contact author: philippa.duckett.21@ucl.ac.uk
  • ‡Contact author: max.hart.22@ucl.ac.uk
  • §Contact author: nikita.pond.18@ucl.ac.uk
  • ∥Contact author: sebastien.rettie@cern.ch
  • Contact author: g.facini@ucl.ac.uk
  • **Contact author: timothy.scanlon@ucl.ac.uk
PDF

Phys. Rev. X 15, 041046 – Published 9 December, 2025

DOI: https://doi.org/10.1103/md46-yqgd

Abstract

Charged particle reconstruction, the identification and characterization of particles from collision data, is fundamental to nearly all research at particle colliders like the Large Hadron Collider (LHC). With the High-Luminosity upgrade (HL-LHC), particle multiplicities will increase substantially, overwhelming traditional track reconstruction algorithms and presenting computational bottlenecks. Here, we introduce a proof of concept for a powerful new method for charged particle reconstruction inspired by state-of-the-art machine learning (ML) approaches in computer vision. Our model leverages transformer neural networks to efficiently filter relevant signals and fully reconstruct particle trajectories, directly tackling the computational complexity that traditional methods face. Evaluated on the widely used TrackML dataset, our approach achieves state-of-the-art tracking efficiency (97%) and a low fake rate (0.7%), requiring just 97 ms to reconstruct on average 1300 particle trajectories from 55,000 detector hits for particles with transverse momentum above 750 MeV. These results represent a significant milestone in both performance and speed, demonstrating a shift toward unified, scalable ML solutions that offer substantial improvements for collider experiments.

View figure in article

Physics Subject Headings (PhySH)

Popular Summary

Experiments at the Large Hadron Collider produce vast volumes of data as charged particles collide and scatter through intricate detector systems. Reconstructing the trajectories of these particles—an essential process known as track reconstruction—is crucial for identifying new particles and probing the basic laws of nature. However, as the collider prepares for a major upgrade, the data will become more complex, pushing current reconstruction methods beyond their practical limits. Our study introduces a new approach that meets this challenge using advanced transformer-based architectures originally developed for computer vision.

We design a single, unified model that performs all key steps of track reconstruction—hit filtering, track finding, and parameter estimation—within one framework. By applying this model to the TrackML dataset—a publicly available simulation of collider data used for benchmarking—we demonstrate that it achieves exceptional accuracy and speed, reconstructing tracks across multiple detector layers in just fractions of a second. Compared with traditional algorithms and earlier machine-learning techniques, this approach offers greater scalability, enabling it to handle the massive data rates expected from next-generation particle physics experiments.

Our findings highlight the potential of transformer architectures to transform how we process and interpret particle collision data. By extending these methods to integrate multiple reconstruction tasks, we can envision more comprehensive systems that accelerate discoveries in high-energy physics. The adaptability and efficiency of this framework make it a promising foundation for future experiments, ultimately helping us explore the most fundamental structures of the Universe.

Article Text

References (81)

  1. L. Evans and P. Bryant, LHC machine, J. Instrum. 3, S08001 (2008).
  2. G. Aad et al. (ATLAS Collaboration), Combination of searches for Higgs boson pairs in pp collisions at s=13  TeV with the ATLAS detector, Phys. Lett. B 800, 135103 (2020).
  3. A. Hayrapetyan et al. (CMS Collaboration), Constraints on the Higgs boson self-coupling from the combination of single and double Higgs boson production in proton-proton collisions at s=13  TeV, Phys. Lett. B 861, 139210 (2025).
  4. G. Aad et al. (ATLAS Collaboration), Electron and photon efficiencies in LHC run 2 with the ATLAS experiment, J. High Energy Phys. 05 (2024) 162.
  5. G. Aad et al. (ATLAS Collaboration), Muon reconstruction and identification efficiency in ATLAS using the full run 2pp collision data set at s=13  TeV, Eur. Phys. J. C 81, 578 (2021).
  6. A. M. Sirunyan et al. (CMS Collaboration), Electron and photon reconstruction and identification with the CMS experiment at the CERN LHC, J. Instrum. 16, P05014 (2021).
  7. A. M. Sirunyan et al. (CMS Collaboration), Performance of the CMS muon detector and muon reconstruction with proton-proton collisions at s=13  TeV, J. Instrum. 13, P06015 (2018).
  8. ATLAS Collaboration, Reconstruction, identification, and calibration of hadronically decaying tau leptons with the ATLAS detector for the LHC run 3 and reprocessed run 2 data, Technical Report, CERN, Geneva, 2022, https://cds.cern.ch/record/2827111/.
  9. A. M. Sirunyan et al. (CMS Collaboration), Performance of reconstruction and identification of τ leptons decaying to hadrons and ντ in pp collisions at s=13  TeV, J. Instrum. 13, P10005 (2018).
  10. M. Aaboud et al. (ATLAS Collaboration), Jet reconstruction and performance using particle flow with the ATLAS Detector, Eur. Phys. J. C 77, 466 (2017).
  11. A. M. Sirunyan et al. (CMS Collaboration), Particle-flow reconstruction and global event description with the CMS detector, J. Instrum. 12, P10003 (2017).
  12. G. Aad et al. (ATLAS Collaboration), ATLAS flavour-tagging algorithms for the LHC run 2pp collision dataset, Eur. Phys. J. C 83, 681 (2023).
  13. G. Aad et al. (ATLAS Collaboration), Transforming jet flavour tagging at ATLAS, arXiv:2505.19689.
  14. A. M. Sirunyan et al. (CMS Collaboration), Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV, J. Instrum. 13, P05011 (2017).
  15. ATLAS Collaboration, ATLAS HL-LHC computing conceptual design report, Technical Report, CERN, 2020, https://cds.cern.ch/record/2729668.
  16. CMS Offline Software and Computing, CMS Phase-2 computing model: Update document, Technical Report, CERN, Geneva, 2022, https://cds.cern.ch/record/2815292.
  17. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems 30 (NeurIPS 2017), edited by I. Guyon, U. von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett (Curran Associates, Inc., Red Hook, NY, 2017), pp. 5998–6008, .
  18. B. Cheng, A. G. Schwing, and A. Kirillov, Per-pixel classification is not all you need for semantic segmentation, in Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS ’21) (Neural Information Processing Systems, San Diego CA, 2021).
  19. S. Van Stroud, N. Pond, M. Hart, J. Barr, S. Rettie, G. Facini, and T. Scanlon, Secondary vertex reconstruction with MaskFormers, Eur. Phys. J. C 84, 1020 (2024).
  20. S. Amrouche et al., The tracking machine learning challenge: Throughput phase, Comput. Software Big Sci. 7, 1 (2023).
  21. A. A. Alves, Jr. et al. (LHCb Collaboration), The LHCb detector at the LHC, J. Instrum. 3, S08005 (2008).
  22. K. Aamodt et al. (ALICE Collaboration), The ALICE experiment at the CERN LHC, J. Instrum. 3, S08002 (2008).
  23. L. Vigani and T. Rudzki (Mu3e Collaboration), The Mu3e detector, J. Instrum. 17, C05024 (2022).
  24. H. Abramowicz et al. (LUXE Collaboration), Technical design report for the LUXE experiment, Eur. Phys. J. Special Topics 233, 1709 (2024).
  25. X. Ai et al., A common tracking software project, Comput. Software Big Sci. 6, 8 (2022).
  26. R. Frühwirth, Application of Kalman filtering to track and vertex fitting, Nucl. Instrum. Methods Phys. Res., Sect. A 262, 444 (1987).
  27. D. Decamp et al. (ALEPH Collaboration), ALEPH: A detector for electron-positron annihilations at LEP, Nucl. Instrum. Methods Phys. Res., Sect. A 294, 121 (1990).
  28. P. Aarnio et al. (DELPHI Collaboration), The DELPHI detector at LEP, Nucl. Instrum. Methods Phys. Res., Sect. A 303, 233 (1991).
  29. G. Aad et al. (ATLAS Collaboration), Software performance of the ATLAS track reconstruction for LHC run 3, Comput. Software Big Sci. 8, 9 (2023).
  30. S. Chatrchyan et al. (CMS Collaboration), Description and performance of track and primary-vertex reconstruction with the CMS tracker, J. Instrum. 9, P10009 (2014).
  31. R. Aaij et al. (LHCb Collaboration), Design and performance of the LHCb trigger and full real-time reconstruction in run 2 of the LHC, J. Instrum. 14, P04013 (2019).
  32. Y. Belikov, M. Ivanov, K. Safarik, and J. Bracinik, TPC tracking and particle identification in high density environment, eConf C 0303241, TULT011 (2003).
  33. ATLAS Collaboration, Fast track reconstruction for HL-LHC, Technical Report, CERN, Geneva, 2019, https://cds.cern.ch/record/2693670.
  34. ATLAS Collaboration, Technical design report for the phase-II upgrade of the ATLAS trigger and data acquisition system—event filter tracking amendment, Technical Report, CERN, Geneva, 2022, https://cds.cern.ch/record/2802799.
  35. CMS Collaboration, The phase-2 upgrade of the CMS level-1 trigger, Technical Report, CERN, Geneva, 2020, final version, https://cds.cern.ch/record/2714892.
  36. ATLAS Collaboration, Improvements and current status of the ACTS-based ITk main pass reconstruction, ATLAS Public Note IDTR-2025-04, 2025, https://atlas.web.cern.ch/Atlas/GROUPS/PHYSICS/PLOTS/IDTR-2025-04/.
  37. J. von Neumann, Theory of Self-Reproducing Automata, edited by A. W. Burks (University of Illinois Press, Urbana, IL, 1966).
  38. C. Verkerk and W. WojcikProceedings of the 10th International Conference on Computing in High-Energy Physics (CHEP ’92), edited by C. Verkerk and W. Wojcik (CERN, Annecy, France, 1992).
  39. A. Glazov, I. Kisel, E. Konotopskaya, and G. Ososkov, Filtering tracks in discrete detectors using a cellular automaton, Nucl. Instrum. Methods Phys. Res., Sect. A 329, 262 (1993).
  40. M. P. Bussa, L. Fava, L. Ferrero, A. Grasso, and V. V. e. a. Ivanov, On a possible second level trigger for the experiment disto, Nuovo Cimento A 109, 327 (1996).
  41. I. Kisel, V. Kovalenko, F. Laplanche, R. Arnold, and C. e. a. Augier, Cellular automaton and elastic net for event reconstruction in the NEMO-2 experiment, Nucl. Instrum. Methods Phys. Res., Sect. A 387, 433 (1997).
  42. A. Bocci, V. Innocente, M. Kortelainen, F. Pantaleo, and M. Rovere, Heterogeneous reconstruction of tracks and primary vertices with the CMS pixel tracker, Front. Big Data 3, 601728 (2020).
  43. CMS Collaboration, Performance of track reconstruction at the CMS high-level trigger in 2022 data, CMS detector performance summary CMS-DP-2023-028, 2023, https://cds.cern.ch/record/2860207.
  44. CMS Collaboration, The phase-2 upgrade of the CMS tracker, Technical Report, CERN, Geneva, 2017. https://cds.cern.ch/record/2272264.
  45. D. Rohr, S. Gorbunov, and V. Lindenstruth (ALICE Collaboration), GPU-accelerated track reconstruction in the ALICE High Level Trigger, J. Phys. Conf. Ser. 898, 032030 (2017).
  46. P. V. C. Hough, Method and means for recognizing complex patterns, U.S. Patent No. 33,069,654 (1962).
  47. R. Aaij et al., Allen: A high level trigger on GPUs for LHCb, Comput. Software Big Sci. 4, 7 (2020).
  48. S. C. Amrouche et al., The tracking machine learning challenge: Accuracy phase, The NeurIPS ’18 Competition (Springer, Cham, 2019).
  49. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, The graph neural network model, IEEE Trans. Neural Networks 20, 61 (2009).
  50. C. Rougier, A. Vallier, D. T. Murnane, J. Stark, P. Calafiura, S. Farrell, S. Caillou, and X. Ju, CTD2022: ATLAS ITk Track Reconstruction with a GNN-based Pipeline, Connecting the Dots Workshop 2022 (CTD2022) (Zenodo, Geneva Switzerland, 2023).
  51. S. Caillou, P. Calafiura, X. Ju, D. Murnane, T. Pham, C. Rougier, J. Stark, and A. Vallier, Physics performance of the ATLAS GNN4ITk track reconstruction chain, EPJ Web Conf. 295, 03030 (2024).
  52. X. Ju et al., Performance of a geometric deep learning pipeline for HL-LHC particle tracking, Eur. Phys. J. C 81, 876 (2021).
  53. C. Biscarat, S. Caillou, C. Rougier, J. Stark, and J. Zahreddine, Towards a realistic track reconstruction algorithm based on graph neural networks for the HL-LHC, EPJ Web Conf. 251, 03047 (2021).
  54. S. Caillou et al., Novel fully-heterogeneous GNN designs for track reconstruction at the HL-LHC, in Proceedings of the 26th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2023) (2023), Vol. 295, p. 09028, 10.1051/epjconf/202329509028.
  55. ATLAS Collaboration, Computational performance of the ATLAS ITk GNN track reconstruction pipeline, Technical Report, CERN, Geneva, 2024, https://cds.cern.ch/record/2914282.
  56. R. Liu, P. Calafiura, S. Farrell, X. Ju, D. T. Murnane, and T. M. Pham, Hierarchical graph neural networks for particle track reconstruction, in Proceedings of the 21st International Workshop on Advanced Computing and Analysis Techniques in Physics Research: AI meets Reality (2023), arXiv:2303.01640.
  57. K. Lieret, G. DeZoort, D. Chatterjee, J. Park, S. Miao, and P. Li, High pileup particle tracking with object condensation, arXiv:2312.03823.
  58. S. Caron, N. Dobreva, A. Ferrer Sánchez, J. D. Martín-Guerrero, U. Odyurt, R. Ruiz de Austri Bazan, Z. Wolffs, and Y. Zhao, TrackFormers: In search of transformer-based particle tracking for the high-luminosity LHC era, arXiv:2407.07179.
  59. A. Huang, Y. Melkani, P. Calafiura, A. Lazar, D. T. Murnane, M.-T. Pham, and X. Ju, A language model for particle tracking, arXiv:2402.10239.
  60. Y. Melkani and X. Ju, TrackSorter: A transformer-based sorting algorithm for track finding in high energy physics, arXiv:2407.21290.
  61. S. Miao, Z. Lu, M. Liu, J. Duarte, and P. Li, Locality-sensitive hashing-based efficient point transformer with applications in high-energy physics, arXiv:2402.12535.
  62. J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi, You only look once: Unified, real-time object detection, arXiv:1506.02640.
  63. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, End-to-end object detection with transformers, arXiv:2005.12872.
  64. K. He, G. Gkioxari, P. Dollár, and R. Girshick, Mask R-CNN, arXiv:1703.06870.
  65. B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, Masked-attention mask transformer for universal image segmentation, in Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Computer Vision and Pattern Recognition, 2021), pp. 1280–1289.
  66. M. Kiehn, S. Amrouche, P. Calafiura, V. Estrade, S. Farrell, C. Germain, V. Gligorov, T. Golling, H. Gray, I. Guyon, M. Hushchyn, V. Innocente, E. Moyse, D. Rousseau, A. Salzburger, A. Ustyuzhanin, J.-R. Vlimant, and Y. Yilnaz, The TrackML high-energy physics tracking challenge on Kaggle, Eur. Phys. J. Web Conf. 214, 06037 (2019).
  67. I. Beltagy, M. E. Peters, and A. Cohan, Longformer: The long-document transformer, arXiv:2004.05150.
  68. T. Dao, FlashAttention-2: Faster attention with better parallelism and work partitioning, arXiv:2307.08691.
  69. N. Shazeer, GLU variants improve transformer, arXiv:2002.05202.
  70. H. Lee, C. Homeyer, R. Herzog, J. Rexilius, and C. Rother, Spatio-temporal outdoor lighting aggregation on image sequences using transformer networks, Int. J. Comput. Vis. 131, 1060 (2022).
  71. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollár, and R. B. Girshick, Segment anything, in Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (Curran Associates, New York, 2023), pp. 3992–4003.
  72. M. Aaboud et al. (ATLAS Collaboration), Performance of the ATLAS track reconstruction algorithms in dense environments in LHC run 2, Eur. Phys. J. C 77, 673 (2017).
  73. R. Girshick, Fast R-CNN, arXiv:1504.08083.
  74. C. H. Sudre, W. Li, T. K. M. Vercauteren, S. Ourselin, and M. J. Cardoso, Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Quebec City, QC (Springer, Cham, 2017), p. 240.
  75. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, Focal loss for dense object detection, IEEE Trans. Pattern Anal. Mach. Intell. 42, 318 (2020).
  76. S. Guthe and D. Thuerck, Algorithm 1015: A fast scalable solver for the dense linear (sum) assignment problem, ACM Trans. Math. Softw. 47, 1 (2021).
  77. M. Hansroul, H. Jeremie, and D. Savard, Fast circle fit with the conformal mapping method, Nucl. Instrum. Methods Phys. Res., Sect. A 270, 498 (1988).
  78. P. J. Fox, S. Huang, J. Isaacson, X. Ju, and B. Nachman, Beyond 4D tracking: Using cluster shapes for track seeding, J. Instrum. 16, P05001 (2021).
  79. R. Liu, P. Calafiura, S. Farrell, X. Ju, D. T. Murnane, and T. M. Pham, Hierarchical graph neural networks for particle track reconstruction, in Proceedings of the 21st International Workshop on Advanced Computing and Analysis Techniques in Physics Research: AI meets Reality (2023), arXiv:2303.01640.
  80. Z. Cao, Z. Liu, D. Luo, H. Zhou, Z. Bu, T. Zhang, T. Zhao, M. R. Lyu, Z. Lin, B. Li et al., Systematic bias of machine learning regression models and correction, arXiv:2405.15950.
  81. J. D. Bossio Sola et al. (ATLAS Collaboration), Technical design report for the ATLAS inner tracker strip detector, Technical Report, CERN, Geneva, 2017.

Outline

Information

Phys. Rev. X 15, 041046– Published 9 December, 2025
  • Received 28 November 2024
  • Revised 21 July 2025
  • Accepted 23 October 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation