Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Transformer networks for continuous gravitational-wave searches

Prasanna. M. Joshi* and Reinhard Prix

  • *Contact author: prasanna.mohan.joshi@aei.mpg.de

Phys. Rev. D 112, 123052 – Published 29 December, 2025

DOI: https://doi.org/10.1103/xgz9-qdkg

Abstract

Wide-parameter-space searches for continuous gravitational waves using semicoherent matched-filter methods require enormous computing power, which limits their achievable sensitivity. Here we explore an alternative search method based on training neural networks as classifiers on detector strain data with minimal preprocessing. Contrary to previous studies using convolutional neural networks (CNNs), we investigate the suitability of the transformer architecture, specifically the vision transformer (ViT). We establish sensitivity benchmarks using the matched-filter F-statistic for ten targeted searches over a ten day timespan, and ten directed and six all-sky searches over a one day timespan. We train ViTs on each of these benchmark cases. The trained ViTs achieve essentially matched-filter sensitivity on the targeted benchmarks, and approach the F-statistic detection probability of pdet=90% on the directed (pdet≈85–89%) and all-sky benchmarks (pdet≈78–88%). Unlike the CNNs in our previous studies, which required extensive manual design and hyperparameter tuning, the ViT achieves better performance with a standard architecture and minimal tuning.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (38)

  1. J. Aasi et al. (LIGO Scientific Collaboration), Advanced LIGO, Classical Quantum Gravity 32, 074001 (2015).
  2. F. Acernese et al. (Virgo Collaboration), Advanced Virgo: A second-generation interferometric gravitational wave detector, Classical Quantum Gravity 32, 024001 (2014).
  3. T. Akutsu et al. (KAGRA Collaboration), Overview of KAGRA: Detector design and construction history, Prog. Theor. Exp. Phys. 2021, 05A101 (2020).
  4. K. Riles, Searches for continuous-wave gravitational radiation, Living Rev. Relativity 26, 3 (2023).
  5. K. Wette, Searches for continuous gravitational waves from neutron stars: A twenty-year retrospective, Astropart. Phys. 153, 102880 (2023).
  6. B. Beheshtipour and M. A. Papa, Deep learning for clustering of continuous gravitational wave candidates, Phys. Rev. D 101, 064009 (2020).
  7. B. Beheshtipour and M. A. Papa, Deep learning for clustering of continuous gravitational wave candidates. II. Identification of low-snr candidates, Phys. Rev. D 103, 064027 (2021).
  8. F. Morawski, M. Bejger, and P. Ciecieląg, Convolutional neural network classifier for the output of the time-domain F-statistic all-sky search for continuous gravitational waves, Mach. Learn. 1, 025016 (2020).
  9. T. S. Yamamoto and T. Tanaka, Use of an excess power method and a convolutional neural network in an all-sky search for continuous gravitational waves, Phys. Rev. D 103, 084049 (2021).
  10. J. Bayley, C. Messenger, and G. Woan, Generalized application of the viterbi algorithm to searches for continuous gravitational-wave signals, Phys. Rev. D 100, 023006 (2019).
  11. J. Bayley, C. Messenger, and G. Woan, Rapid parameter estimation for an all-sky continuous gravitational wave search using conditional variational auto-encoders, Phys. Rev. D 106, 083022 (2022).
  12. J. Bayley, C. Messenger, and G. Woan, Robust machine learning algorithm to search for continuous gravitational waves, Phys. Rev. D 102, 083024 (2020).
  13. T. S. Yamamoto, A. L. Miller, M. Sieniawska, and T. Tanaka, Assessing the impact of non-Gaussian noise on convolutional neural networks that search for continuous gravitational waves, Phys. Rev. D 106, 024025 (2022).
  14. R. Tenorio, M. J. Williams, J. Bayley, C. Messenger, M. Demkin, W. Reade, and K. Competitors, Learning to detect continuous gravitational waves: An open data-analysis competition (2025), 10.1088/2632-2153/ae2dbc.
  15. A. L. Miller, P. Astone, S. D’Antonio, S. Frasca, G. Intini, I. La Rosa, P. Leaci, S. Mastrogiovanni, F. Muciaccia, A. Mitidis, C. Palomba, O. J. Piccinni, A. Singhal, B. F. Whiting, and L. Rei, How effective is machine learning to detect long transient gravitational waves from neutron stars in a real search?, Phys. Rev. D 100, 062005 (2019).
  16. L. M. Modafferi, R. Tenorio, and D. Keitel, Convolutional neural network search for long-duration transient gravitational waves from glitching pulsars, Phys. Rev. D 108, 023005 (2023).
  17. F. Attadio, L. Ricca, M. Serra, C. Palomba, P. Astone, S. Dall’Osso, S. Dal Pra, S. D’Antonio, M. Di Giovanni, L. D’Onofrio, P. Leaci, F. Muciaccia, L. Pierini, and F. Safai Tehrani, Neural network method to search for long transient gravitational waves, Phys. Rev. D 110, 103047 (2024).
  18. C. Dreissigacker, R. Sharma, C. Messenger, R. Zhao, and R. Prix, Deep-learning continuous gravitational waves, Phys. Rev. D 100, 044009 (2019).
  19. C. Dreissigacker and R. Prix, Deep-learning continuous gravitational waves: Multiple detectors and realistic noise, Phys. Rev. D 102, 022005 (2020).
  20. P. M. Joshi and R. Prix, Novel neural-network architecture for continuous gravitational waves, Phys. Rev. D 108, 063021 (2023).
  21. P. M. Joshi and R. Prix, Large-kernel convolutional neural networks for wide parameter-space searches of continuous gravitational waves, Phys. Rev. D 110, 124071 (2024).
  22. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, arXiv:1706.03762.
  23. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An image is worth 16×16 words: Transformers for image recognition at scale, arXiv:2010.11929.
  24. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, arXiv:2103.14030.
  25. Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, A convnet for the 2020s, arXiv:2201.03545.
  26. T. Shehzadi, K. A. Hashmi, D. Stricker, and M. Z. Afzal, Object detection with transformers: A review, arXiv:2306.04670.
  27. X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, Deformable detr: Deformable transformers for end-to-end object detection, arXiv:2010.04159.
  28. P. Jaranowski, A. Królak, and B. F. Schutz, Data analysis of gravitational-wave signals from spinning neutron stars: The signal and its detection, Phys. Rev. D 58, 063001 (1998).
  29. C. Dreissigacker, R. Prix, and K. Wette, Fast and accurate sensitivity estimation for continuous-gravitational-wave searches, Phys. Rev. D 98, 084058 (2018).
  30. K. Wette, Estimating the sensitivity of wide-parameter-space searches for gravitational-wave pulsars, Phys. Rev. D 85, 042003 (2012).
  31. K. Wette, S. Walsh, R. Prix, and M. A. Papa, Implementing a semicoherent search for continuous gravitational waves using optimally constructed template banks, Phys. Rev. D 97, 123016 (2018).
  32. L. S. Collaboration, SFT data format version 2 specification, Technical Report No. T040164, 2004, https://dcc.ligo.org/LIGO-T040164/public.
  33. M. B. Schäfer, O. c. v. Zelenka, A. H. Nitz, F. Ohme, and B. Brügmann, Training strategies for deep learning gravitational-wave searches, Phys. Rev. D 105, 043002 (2022).
  34. D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980.
  35. M. Abadi et al., tensorflow: Large-scale machine learning on heterogeneous systems (2015), software available from tensorflow.org.
  36. F. Chollet et al., keras, https://keras.io (2015).
  37. L. Biewald, Experiment tracking with weights and biases (2020), software available from wandb.com.
  38. N. Nagarajan and C. Messenger, Identifying and mitigating machine learning biases for the gravitational-wave detection problem, Phys. Rev. D 112, 103002 (2025).

Outline

Information

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation