- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Enhancing the reliability of machine learning for gravitational wave parameter estimation with attention-based models

Hibiki Iwanaga*, Mahoro Matsuyama, and Yousuke Itoh

  • *Contact author: iwanagahibiki7382@gmail.com
PDF

Phys. Rev. D 112, 083020 – Published 9 October, 2025

DOI: https://doi.org/10.1103/blfk-7k9f

Abstract

We introduce a technique to enhance the reliability of gravitational wave parameter estimation results produced by machine learning. We develop two independent machine learning models based on the Vision Transformer to estimate effective spin and chirp mass from spectrograms of gravitational wave signals from binary black hole mergers. To enhance the reliability of these models, we utilize attention maps to visualize the areas our models focus on when making predictions. This approach enables us to demonstrate that both models perform parameter estimation based on physically meaningful information. Furthermore, by leveraging these attention maps, we demonstrate a method to quantify the impact of glitches on parameter estimation. We show that as the models focus more on glitches, the parameter estimation results become more strongly biased. This suggests that attention maps could potentially be used to distinguish between cases where the results produced by the machine learning model are reliable and cases where they are not.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (61)

  1. B. Abbott et al. (The LIGO Scientific Collaboration and the Virgo Collaboration), Observation of gravitational waves from a binary black hole merger, Phys. Rev. Lett. 116, 061102 (2016).
  2. J. Aasi et al. (The LIGO Scientific Collaboration), Advanced LIGO, Classical Quantum Gravity 32, 074001 (2015).
  3. F. Acernese et al., Advanced Virgo: A second-generation interferometric gravitational wave detector, Classical Quantum Gravity 32, 024001 (2015).
  4. B. Abbott et al. (The LIGO Scientific Collaboration and the Virgo Collaboration), GWTC-1: A gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs, Phys. Rev. X 9, 031040 (2019).
  5. R. Abbott et al. (The LIGO Scientific Collaboration and the Virgo Collaboration), GWTC-2: Compact binary coalescences observed by LIGO and Virgo during the first half of the third observing run, Phys. Rev. X 11, 021053 (2021).
  6. R. Abbott et al., GWTC-3: Compact binary coalescences observed by LIGO and Virgo during the second part of the third observing run, Phys. Rev. X 13, 041039 (2023).
  7. Y. Aso, Y. Michimura, K. Somiya, M. Ando, O. Miyakawa, T. Sekiguchi, D. Tatsumi, and H. K. Yamamoto, Interferometer design of the KAGRA gravitational wave detector, Phys. Rev. D 88, 043007 (2013).
  8. C. S. Unnikrishnan, IndIGO and Ligo-India scope and plans for gravitational wave research and precision metrology in India, Int. J. Mod. Phys. D 22, 1341010 (2013).
  9. B. Abbott et al., Prospects for observing and localizing gravitational-wave transients with Advanced LIGO, Advanced Virgo and KAGRA, Living Rev. Relativity 23, 3 (2020).
  10. B. P. Abbott et al., A guide to LIGO–Virgo detector noise and extraction of transient gravitational-wave signals, Classical Quantum Gravity 37, 055002 (2020).
  11. D. Foreman-Mackey, D. W. Hogg, D. Lang, and J. Goodman, emcee: The MCMC Hammer, Publ. Astron. Soc. Pac. 125, 306 (2013).
  12. J. S. Speagle, DYNESTY: A dynamic nested sampling package for estimating Bayesian posteriors and evidences, Mon. Not. R. Astron. Soc. 493, 3132 (2020).
  13. J. Veitch et al., Parameter estimation for compact binaries with ground-based gravitational-wave observations using the LALInference software library, Phys. Rev. D 91, 042003 (2015).
  14. G. Ashton et al., bilby: A user-friendly bayesian inference library for gravitational-wave astronomy, Astrophys. J. 241, 27 (2019).
  15. B. Abbott et al., Properties of the binary neutron star merger GW170817, Phys. Rev. X 9, 011001 (2019).
  16. F. S. Broekgaarden, S. Banagiri, and E. Payne, Visualizing the number of existing and future gravitational-wave detections from merging double compact objects, Astrophys. J. 969, 108 (2024).
  17. A. McLeod, D. Jacobs, C. Chatterjee, L. Wen, and F. Panther, Rapid mass parameter estimation of binary black hole coalescences using deep learning, arXiv:2201.11126.
  18. A. J. Chua and M. Vallisneri, Learning bayesian posteriors with neural networks for gravitational-wave inference, Phys. Rev. Lett. 124, 041102 (2020).
  19. H. Gabbard, C. Messenger, I. S. Heng, F. Tonolini, and R. Murray-Smith, Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy, Nat. Phys. 18, 112 (2022).
  20. S. R. Green and J. Gair, Complete parameter inference for GW150914 using deep learning, Mach. Learn. Sci. Tech. 2, 03LT01 (2021).
  21. S. R. Green, C. Simpson, and J. Gair, Gravitational-wave parameter estimation with autoregressive neural network flows, Phys. Rev. D 102, 104057 (2020).
  22. H. Shen, E. A. Huerta, E. O’Shea, P. Kumar, and Z. Zhao, Statistically-informed deep learning for gravitational wave parameter estimation, Mach. Learn. Sci. Technol. 3, 015007 (2022).
  23. M. Dax, S. R. Green, J. Gair, J. H. Macke, A. Buonanno, and B. Schölkopf, Real-time gravitational wave science with neural posterior estimation, Phys. Rev. Lett. 127, 241103 (2021).
  24. M. Dax, S. R. Green, J. Gair, N. Gupte, M. Pürrer, V. Raymond, J. Wildberger, J. H. Macke, A. Buonanno, and B. Schölkopf, Real-time inference for binary neutron star mergers using machine learning, Nature (London) 639, 49 (2025).
  25. P. Couvares, I. Bird, E. Porter, S. Bagnasco, M. Punturo, D. Reitze, S. Katsanevas, T. Kajita, V. Kalogera, H. Lueck, D. McClelland, S. Rowan, G. Sanders, B. S. Sathyaprakash, D. Shoemaker, and J. van den Brand, Gravitational wave data analysis: Computing challenges in the 3G Era, arXiv:2111.06987.
  26. E. Cuoco et al., Enhancing gravitational-wave science with machine learning, Mach. Learn. Sci. Technol. 2, 011002 (2021).
  27. Y. Sakai, Y. Itoh, P. Jung, K. Kokeyama, C. Kozakai, K. T. Nakahira, S. Oshino, Y. Shikano, H. Takahashi, T. Uchiyama, G. Ueshima, T. Washimi, T. Yamamoto, and T. Yokozawa, Training process of unsupervised learning architecture for gravity spy dataset, Ann. Phys. (Berlin) 536, 2200140 (2024).
  28. Y. Sakai, Y. Itoh, P. Jung, K. Kokeyama, C. Kozakai, K. T. Nakahira, S. Oshino, Y. Shikano, H. Takahashi, T. Uchiyama, G. Ueshima, T. Washimi, T. Yamamoto, and T. Yokozawa, Unsupervised learning architecture for classifying the transient noise of interferometric gravitational-wave detectors, Sci. Rep. 12, 9935 (2022).
  29. S. Morisaki, Accelerating parameter estimation of gravitational waves from compact binary coalescence using adaptive frequency resolutions, Phys. Rev. D 104, 044062 (2021).
  30. E. Lee, S. Morisaki, and H. Tagoshi, Mass-spin reparametrization for a rapid parameter estimation of inspiral gravitational-wave signals, Phys. Rev. D 105, 124057 (2022).
  31. N. Adhikari and S. Morisaki, Accelerating gravitational-wave parametrized tests of general relativity using a multiband decomposition of likelihood, Phys. Rev. D 106, 104053 (2022).
  32. S. Morisaki, R. Smith, L. Tsukada, S. Sachdev, S. Stevenson, C. Talbot, and A. Zimmerman, Rapid localization and inference on compact binary coalescences with the Advanced LIGO-Virgo-KAGRA gravitational-wave detector network, Phys. Rev. D 108, 123040 (2023).
  33. M. Tulio Ribeiro, S. Singh, and C. Guestrin, in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Demonstrations Session (Association for Computing Machinery, New York, NY, 2016), pp. 97–101.
  34. Z. C. Lipton, in Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016) (Association for Computing Machinery, New York, NY, 2016).
  35. H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, Training data-efficient image transformers & distillation through attention, arXiv:2012.12877.
  36. A. Dosovitskiy et al., An image is worth 16×16 words: Transformers for image recognition at scale, arXiv:2010.11929.
  37. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, in Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (Curran Associates Inc., Red Hook, NY, 2017).
  38. G. Hinton, O. Vinyals, and J. Dean, in Proceedings of the NIPS 2014 Deep Learning Workshop (Curran Associates Inc., Red Hook, NY, 2015).
  39. A. Nitz et al., gwastro/pycbc: v2.3.3 release of pycbc, 10.5281/zenodo.10473621 (2024).
  40. C. Devine, Z. B. Etienne, and S. T. McWilliams, Optimizing spinning time-domain gravitational waveforms for advanced LIGO data analysis, Classical Quantum Gravity 33, 125025 (2016).
  41. S. Chatterji, L. Blackburn, G. Martin, and E. Katsavounidis, Multiresolution techniques for the detection of gravitational-wave bursts, Classical Quantum Gravity 21, S1809 (2004).
  42. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (, New York, NY, 2009), pp. 248–255.
  43. D. P. Kingma and J. Ba, in Proceedings of the 3rd International Conference for Learning Representations, San Diego, 2015 (The International Conference on Learning Representations (ICLR), Appleton, WI, 2014).
  44. B. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), Properties of the binary black hole merger GW150914, Phys. Rev. Lett. 116, 241102 (2016).
  45. L. Blanchet, Post-Newtonian theory for gravitational waves, Living Rev. Relativity 27, 4 (2024).
  46. I. Harry and T. Hinderer, Observing and measuring the neutron-star equation-of-state in spinning binary neutron star systems, Classical Quantum Gravity 35, 145010 (2018).
  47. D. Davis et al., LIGO detector characterization in the second and third observing runs, Classical Quantum Gravity 38, 135014 (2021).
  48. F. Acernese et al., Virgo detector characterization and data quality during the O3 run, arXiv:2205.01555.
  49. T. Akutsu et al., Overview of KAGRA: Calibration, detector characterization, physical environmental monitors, and the geophysics interferometer, Prog. Theor. Exp. Phys. 2021, 05A102 (2021).
  50. B. Abbott et al., Effects of data quality vetoes on a search for compact binary coalescences in Advanced LIGO’s first observing run, Classical Quantum Gravity 35, 065010 (2018).
  51. D. Davis, T. B. Littenberg, I. M. Romero-Shaw, M. Millhouse, J. McIver, F. Di Renzo, and G. Ashton, Subtracting glitches from gravitational-wave detector data during the third LIGO-Virgo observing run, Classical Quantum Gravity 39, 245013 (2022).
  52. J. Powell, Parameter estimation and model selection of gravitational wave signals contaminated by transient detector noise glitches, Classical Quantum Gravity 35, 155017 (2018).
  53. R. Macas, J. Pooley, L. K. Nuttall, D. Davis, M. J. Dyer, Y. Lecoeuche, J. D. Lyman, J. McIver, and K. Rink, Impact of noise transients on low latency gravitational-wave event localization, Phys. Rev. D 105, 103021 (2022).
  54. E. Payne, S. Hourihane, J. Golomb, R. Udall, D. Davis, and K. Chatziioannou, Curious case of GW200129: Interplay between spin-precession inference and data-quality issues, Phys. Rev. D 106, 104017 (2022).
  55. R. Udall, S. Hourihane, S. Miller, D. Davis, K. Chatziioannou, M. Isi, and H. Deshong, Antialigned spin of GW191109: Glitch mitigation and its implications, Phys. Rev. D 111, 024046 (2025).
  56. L. Blackburn et al., The LSC glitch group: Monitoring noise transients during the fifth LIGO science run, Classical Quantum Gravity 25, 184004 (2008).
  57. S. Ghonge, J. Brandt, J. M. Sullivan, M. Millhouse, K. Chatziioannou, J. A. Clark, T. Littenberg, N. Cornish, S. Hourihane, and L. Cadonati, Assessing and mitigating the impact of glitches on gravitational-wave parameter estimation: A model agnostic approach, Phys. Rev. D 110, 122002 (2024).
  58. S. Abnar and W. Zuidema, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Association for Computational Linguistics, Kerrville, TX, 2020), pp. 4190–4197.
  59. A. Paszke, S. Gross, F. Massa et al., in Advances in Neural Information Processing Systems 32 (Curran Associates, Inc., Red Hook, NY, 2019), pp. 8024–8035.
  60. J. D. Hunter, Matplotlib: A 2D graphics environment, Comput. Sci. Eng. 9, 90 (2007).
  61. https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/deit.py.

Outline

Information

Phys. Rev. D 112, 083020– Published 9 October, 2025
  • Received 22 January 2025
  • Accepted 10 September 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation