- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Identifying and mitigating machine-learning biases for the gravitational-wave detection problem

Narenraju Nagarajan* and Christopher Messenger†

  • *Contact author: n.nagarajan.1@research.gla.ac.uk
  • †Contact author: christopher.messenger@glasgow.ac.uk
PDF

Phys. Rev. D 112, 103002 – Published 4 November, 2025

DOI: https://doi.org/10.1103/zwj9-ycyz

Abstract

Matched filtering is a long-standing technique for the optimal detection of known signals in stationary Gaussian noise. However, it has known departures from optimality when operating on unknown signals in real noise and suffers from computational inefficiencies in its pursuit of near-optimality. A compelling alternative that has emerged in recent years to address this problem is deep learning. Although it has shown significant promise when applied to the search for gravitational-waves in detector noise, we demonstrate the existence of a multitude of learning biases that hinder generalization and lead to significant loss in detection sensitivity. Our work identifies the sources of a set of 11 interconnected biases present in the supervised learning of the gravitational-wave detection problem and contributes mitigation tactics and training strategies to concurrently address them. In light of the identified biases, we demonstrate that existing detection sensitivity metrics are not reliable for machine-learning pipelines and discuss the trustworthiness of previous results. We use gravitational-wave domain knowledge to build a bespoke machine-learning-based binary black hole search pipeline called Sage that addresses these biases. Via the injection study presented in the Machine-Learning Gravitational-Wave Search Challenge, we show that Sage detects ≈11.2% more signals than the benchmark pycbc analysis at a false-alarm rate of one per month in O3a noise. Moreover, we also show that it can detect ≈48.29% more signals than the previous best-performing machine-learning pipeline on the same dataset. We empirically prove that Sage can (i) effectively handle out-of-distribution noise power spectral densities, (ii) strongly reject non-Gaussian transient noise artifacts, and (iii) achieve higher detection sensitivities using less data than network architectures of a similar size. By studying machine-learning biases and conducting empirical investigations to understand the reasons for performance improvement or degradation, we aim to address the need for interpretability of machine-learning methods for gravitational-wave science. All code and implementations are available.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (134)

  1. F. Chollet, arXiv:1911.01547.
  2. Z. Fang, Y. Li, J. Lu, J. Dong, B. Han, and F. Liu, arXiv:2210.14707.
  3. N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, ACM Comput. Surv., 10.1145/3457607 (2021).
  4. I. H. Sarker, Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions (Springer-Verlag, Berlin, Heidelberg, 2021).
  5. S. V. Dhurandhar and B. S. Sathyaprakash, Phys. Rev. D 49, 1707 (1994).
  6. B. J. Owen and B. S. Sathyaprakash, Phys. Rev. D 60, 022002 (1999).
  7. S. Babak et al., Phys. Rev. D 87, 024033 (2013).
  8. S. A. Usman et al., Classical Quantum Gravity 33, 215004 (2016).
  9. A. D. Whalen, IEEE Commun. Mag. 22, 37 (1984).
  10. B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Classical Quantum Gravity 37, 055002 (2020).
  11. R. Abbott et al. (LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration), Phys. Rev. X 13, 041039 (2023).
  12. T. Accadia et al., in Proceedings of the 16th Lomonosov Conference on Elementary Particle Physics (2015), pp. 261–270.
  13. B. P. Abbott et al., Living Rev. Relativity 21, 3 (2018).
  14. LIGO Scientific Collaboration et al., Classical Quantum Gravity 32, 074001 (2015).
  15. A. Buikema et al., Phys. Rev. D 102, 062003 (2020).
  16. M. Tse et al., Phys. Rev. Lett. 123, 231107 (2019).
  17. D. V. Martynov et al., Phys. Rev. D 93, 112004 (2016).
  18. B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), Phys. Rev. Lett. 116, 131103 (2016).
  19. G. M. Harry (for LIGO Scientific Collaboration), Classical Quantum Gravity 27, 084006 (2010).
  20. F. Acernese et al. (Virgo Collaboration), Phys. Rev. Lett. 123, 231108 (2019).
  21. M. B. Schäfer et al., Phys. Rev. D 107, 023021 (2023).
  22. B. Allen, W. G. Anderson, P. R. Brady, D. A. Brown, and J. D. E. Creighton, Phys. Rev. D 85, 122006 (2012).
  23. A. H. Nitz, T. Dent, T. D. Canton, S. Fairhurst, and D. A. Brown, Astrophys. J. 849, 118 (2017).
  24. A. H. Nitz, T. Dal Canton, D. Davis, and S. Reyes, Phys. Rev. D 98, 024050 (2018).
  25. C. Rover, J. Phys. Conf. Ser. 228, 012008 (2010).
  26. A. C. Searle, in Proceedings of the 12th Gravitational Wave Data Analysis Workshop (2008), arXiv:0804.1161.
  27. P. Kumar and T. Dent, Phys. Rev. D 110, 043036 (2024).
  28. T. Dent and J. Veitch, Phys. Rev. D 89, 062002 (2014).
  29. J. Yan, M. Avagyan, R. E. Colgan, D. Veske, I. Bartos, J. Wright, Z. Márka, and S. Márka, Phys. Rev. D 105, 043006 (2022).
  30. S. Babak, R. Balasubramanian, D. Churches, T. Cokelaer, and B. S. Sathyaprakash, Classical Quantum Gravity 23, 5477 (2006).
  31. J. R. Gair, L. Barack, T. Creighton, C. Cutler, S. L. Larson, E. S. Phinney, and M. Vallisneri, Classical Quantum Gravity 21, S1595 (2004).
  32. D. C. Srivastava and S. K. Sahay, Mon. Not. R. Astron. Soc. 337, 322 (2002).
  33. J. Aasi et al. (LIGO Scientific Collaboration), Classical Quantum Gravity 32, 074001 (2015).
  34. B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Classical Quantum Gravity 33, 134001 (2016).
  35. M. Cabero et al., Classical Quantum Gravity 36, 15 (2019).
  36. S. Biscoveanu, C.-J. Haster, S. Vitale, and J. Davies, Phys. Rev. D 102, 023008 (2020).
  37. B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Classical Quantum Gravity 35, 065010 (2018).
  38. C. Messick et al., Phys. Rev. D 95, 042001 (2017).
  39. B. Allen, Phys. Rev. D 71, 062001 (2005).
  40. B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. X 9, 031040 (2019).
  41. B. P. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. D 93, 122003 (2016).
  42. A. H. Nitz, Classical Quantum Gravity 35, 035016 (2018).
  43. R. Balestriero, A. I. Humayun, and R. Baraniuk, arXiv:2408.04809.
  44. H. Ye, G. Y. Li, and B.-H. Juang, IEEE Wireless Commun. Lett. 7, 114 (2018).
  45. J. Chen, Y. Wang, and D. Wang, Language Process. 22, 1993 (2014).
  46. J. Yu, Z. Wang, V. Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu, arXiv:2205.01917.
  47. K. Kawaguchi, L. Pack Kaelbling, and Y. Bengio, arXiv:1710.05468.
  48. G. V. Pérez, C. Q. Camargo, and A. A. Louis, arXiv:1805.08522.
  49. M.-P. Jarabo-Amores, M. Rosa-Zurera, R. Gil-Pita, and F. Lopez-Ferreras, IEEE Trans. Signal Process. 57, 4175 (2009).
  50. M.-P. Jarabo-Amores, D. d. la Mata-Moya, R. Gil-Pita, and M. Rosa-Zurera, EURASIP J. Appl. Signal Process. 2013, 44 (2013).
  51. D. George and E. A. Huerta, Phys. Lett. B 778, 64 (2018).
  52. H. Gabbard, M. Williams, F. Hayes, and C. Messenger, Phys. Rev. Lett. 120, 141103 (2018).
  53. P. Nousi, A. E. Koloniari, N. Passalis, P. Iosif, N. Stergioulas, and A. Tefas, Phys. Rev. D 108, 024022 (2023).
  54. A. E. Koloniari, E. C. Koursoumpa, P. Nousi, P. Lampropoulos, N. Passalis, A. Tefas, and N. Stergioulas, Mach. Learn. Sci. Tech. 6, 015054 (2025).
  55. M. B. Schäfer and A. H. Nitz, Phys. Rev. D 105, 043003 (2022).
  56. E. Marx, W. Benoit, A. Gunny, R. Omer, D. Chatterjee, R. C. Venterea, L. Wills, M. Saleem, E. Moreno, R. Raikman, E. Govorkova, D. Rankin, M. W. Coughlin, P. Harris, and E. Katsavounidis, arXiv:2403.18661.
  57. T. D. Gebhard, N. Kilbertus, I. Harry, and B. Schölkopf, Phys. Rev. D 100, 063015 (2019).
  58. C. Cutler and É. E. Flanagan, Phys. Rev. D 49, 2658 (1994).
  59. C. Mingard, J. Skalse, G. V. Pérez, D. Martínez-Rubio, V. Mikulik, and A. A. Louis, arXiv:1909.11522.
  60. G. V. Pérez, C. Q. Camargo, and A. A. Louis, arXiv:1805.08522.
  61. J. Lee, Y. Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y. W. Teh, arXiv:1810.00825.
  62. N. Natarajan, I. S. Dhillon, P. Ravikumar, and A. Tewari, in Proceedings of the 26th International Conference on Neural Information Processing Systems—Volume 1, NIPS’13 (Curran Associates Inc., Red Hook, NY, 2013), pp. 1196–1204.
  63. M. Denil and T. Trappenberg, in Proceedings of the 23rd Canadian Conference on Advances in Artificial Intelligence, AI’10 (Springer-Verlag, Berlin, Heidelberg, 2010), pp. 220–231.
  64. O. Zelenka, B. Brügmann, and F. Ohme, arXiv:2402.07492.
  65. C. M. Bishop and N. M. Nasrabadi, Pattern Recognition and Machine Learning (Springer, New York, 2006), Vol. 4.
  66. M. S. Masiha, A. Gohari, M. H. Yassaee, and M. R. Aref, in Proceedings of the 2021 IEEE International Symposium on Information Theory (ISIT) (IEEE, New York, 2021), pp. 2912–2917.
  67. R. J. N. Baldock, H. Maennel, and B. Neyshabur, arXiv:2106.09647.
  68. Y. Bengio, P. Simard, and P. Frasconi, IEEE Trans. Neural Networks 5, 157 (1994).
  69. S. E. Hihi and Y. Bengio, in Neural Information Processing Systems (1995), https://api.semanticscholar.org/CorpusID:2843869.
  70. S. Hochreiter and J. Schmidhuber, Neural Comput. 9, 1735 (1997).
  71. A. Vaswani, N. M. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, in Neural Information Processing Systems (2017), https://api.semanticscholar.org/CorpusID:13756489.
  72. W. Luo, Y. Li, R. Urtasun, and R. S. Zemel, arXiv:1701.04128.
  73. N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, in Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research Vol. 97, edited by K. Chaudhuri and R. Salakhutdinov (PMLR, 2019), pp. 5301–5310.
  74. R. Basri, M. Galun, A. Geifman, D. W. Jacobs, Y. Kasten, and S. Kritchman, in Proceedings of the International Conference on Machine Learning (2020).
  75. N. Nagarajan, nnarenraju/sage: First DOI version of Sage, 10.5281/zenodo.17290134.
  76. G. Pratten, C. García-Quirós, M. Colleoni, A. Ramos-Buades, H. Estellés, M. Mateu-Lucena, R. Jaume, M. Haney, D. Keitel, J. E. Thompson, and S. Husa, Phys. Rev. D 103, 104056 (2021).
  77. C. Mills and S. Fairhurst, Phys. Rev. D 103, 024042 (2021).
  78. R. Green, C. Hoy, S. Fairhurst, M. Hannam, F. Pannarale, and C. Thomas, Phys. Rev. D 103, 124023 (2021).
  79. R. Abbott et al. (LIGO Scientific and Virgo Collaborations), Phys. Rev. X 11, 021053 (2021).
  80. https://gwosc.org/datadef/.
  81. J. Solomon and O. M., PSD computations using Welch’s method (1991), https://www.osti.gov/biblio/5688766.
  82. J. Glanzer et al., Gravity spy machine learning classifications of LIGO glitches from observing runs O1, O2, O3A, and O3B (2021), 10.5281/zenodo.5649212.
  83. https://github.com/gwastro/ml-mock-data-challenge-1.
  84. S. Babak, R. Balasubramanian, D. Churches, T. Cokelaer, and B. S. Sathyaprakash, Classical Quantum Gravity 23, 5477 (2006).
  85. B. S. Sathyaprakash and S. V. Dhurandhar, Phys. Rev. D 44, 3819 (1991).
  86. T. Cokelaer, Phys. Rev. D 76, 102004 (2007).
  87. https://gwosc.org/eventapi/html/allevents/.
  88. https://gwosc.org/timeline/show/O3b_4KHZ_R1/H1_DATA*L1_DATA*V1_DATA/1256655618/12708000/.
  89. J. P. Cohen, T. Cao, J. D. Viviano, C.-W. Huang, M. Fralick, M. Ghassemi, M. Mamdani, R. Greiner, and Y. Bengio, Cmaj 193, E1391 (2021).
  90. E. Beede, E. Baylor, F. Hersch, A. Iurchenko, L. Wilcox, P. Ruamviboonsuk, and L. M. Vardoulakis, in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI ’20 (Association for Computing Machinery, New York, NY, 2020), pp. 1–12.
  91. M. Wąs, M.-A. Bizouard, V. Brisson, F. Cavalier, M. Davier, P. Hello, N. Leroy, F. Robinet, and M. Vavoulidis, Classical Quantum Gravity 27, 015005 (2010).
  92. M. B. Schäfer, F. Ohme, and A. H. Nitz, Phys. Rev. D 102, 063015 (2020).
  93. D. P. Kingma and J. Ba, arXiv:1412.6980.
  94. R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, in Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, New York, 2014), p. 580.
  95. K. He, X. Zhang, S. Ren, and J. Sun, IEEE Trans. Pattern Anal. Mach. Intell. 37, 1904 (2015).
  96. K. He, G. Gkioxari, P. Dollár, and R. Girshick, in Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV) (IEEE, New York, 2017), pp. 2980–2988.
  97. C. Messenger, C. Zerafa, E. Cuoco, M. J. Williams, and W. Reade, G2net gravitational wave detection (2021), https://www.kaggle.com/competitions/g2net-gravitational-wave-detection.
  98. C. Peng, X. Zhang, G. Yu, G. Luo, and J. Sun, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2017).
  99. K. Zhou, Y. Yang, A. Cavallaro, and T. Xiang, in Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (IEEE, New York, 2019), p. 3701.
  100. K. He, X. Zhang, S. Ren, and J. Sun, in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2016), pp. 770–778.
  101. S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, in Computer Vision—ECCV 2018: 15th European Conference, Munich, Germany, 2018, Proceedings, Part VII (Springer-Verlag, Berlin, Heidelberg, 2018), pp. 3–19.
  102. D. Bank, N. Koenigstein, and R. Giryes, Deep Learning in Science (2021), https://api.semanticscholar.org/CorpusID:212717965.
  103. J. Hu, L. Shen, and G. Sun, in Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE, New York, 2018), pp. 7132–7141.
  104. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2016), pp. 2921–2929.
  105. V. Tiwari and S. Fairhurst, Astrophys. J. Lett. 913, L19 (2021).
  106. Z. Cui, W. Chen, and Y. Chen, arXiv:1603.06995.
  107. X. Liu, B. Xu, S. Cao, and L. Zhang, J. Comput. Phys. 506, 112944 (2024).
  108. X. Chai, W. Cao, J. Li, H. Long, and X. Sun, IEEE Trans. Geosci. Remote Sens. 62, 1 (2024).
  109. R. Fang and Y. Xu, arXiv:2410.16105.
  110. B. L. Edelman, S. Goel, S. M. Kakade, and C. Zhang, arXiv:2110.10090.
  111. S. Hao, D.-H. Lee, and D. Zhao, Trans. Res. Part C 107, 287 (2019).
  112. https://github.com/gwastro/ml-mock-data-challenge-1/blob/master/evaluate.py.
  113. A. H. Nitz, S. Kumar, Y.-F. Wang, S. Kastha, S. Wu, M. Schäfer, R. Dhurkunde, and C. D. Capano, Astrophys. J. 946, 59 (2023).
  114. D. Davis and M. Walker, Galaxies 10, 12 (2022).
  115. M. Cabero et al., Classical Quantum Gravity 36, 15 (2019).
  116. L. Nuttall et al., Classical Quantum Gravity 32, 245005 (2015).
  117. A. H. Nitz, T. Dent, T. Dal Canton, S. Fairhurst, and D. A. Brown, Astrophys. J. 849, 118 (2017).
  118. M. Cabero et al., Classical Quantum Gravity 36, 15 (2019).
  119. J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, Proc. Natl. Acad. Sci. U.S.A. 114, 3521 (2017).
  120. M. Toneva, A. Sordoni, R. T. des Combes, A. Trischler, Y. Bengio, and G. J. Gordon, arXiv:1812.05159.
  121. M. P. Kumar, B. Packer, and D. Koller, in Proceedings of the 23rd International Conference on Neural Information Processing Systems—Volume 1, NIPS’10 (Curran Associates Inc., Red Hook, NY, 2010), pp. 1189–1197.
  122. F. Robinet, N. Arnaud, N. Leroy, A. Lundgren, D. Macleod, and J. McIver, SoftwareX 12, 100620 (2020).
  123. S. Sahayam, J. Zakkam, and U. Jayaraman, arXiv:2304.03486.
  124. Y. Suh, B. Han, W. Kim, and K. M. Lee, in Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2019), pp. 7244–7252.
  125. R. Yu, Z. Dou, S. Bai, Z. Zhang, Y. Xu, and X. Bai, in Computer Vision—ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XVI (Springer-Verlag, Berlin, Heidelberg, 2018), pp. 196–212.
  126. D. Arpit, S. Jastrzundefinedbski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio, and S. Lacoste-Julien, in Proceedings of the 34th International Conference on Machine Learning—Volume 70, ICML’17 (JMLR.org, 2017), pp. 233–242.
  127. S. Chatterjee, in Proceedings of the International Conference on Machine Learning (PMLR, 2018), pp. 755–763.
  128. X. Wang, Y. Chen, and W. Zhu, IEEE Trans. Pattern Anal. Mach. Intell. 44, 4555 (2022).
  129. R. Meyes, M. Lu, C. W. de Puiseau, and T. Meisen, arXiv:1901.08644.
  130. Y. Dauphin and Y. Bengio, arXiv:1301.3583.
  131. S. Gao, M.-M. Cheng, K. Zhao, X. Zhang, M.-H. Yang, and P. H. S. Torr, IEEE Trans. Pattern Anal. Mach. Intell. 43, 652 (2019).
  132. G. Papamakarios, E. T. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, J. Mach. Learn. Res. 22, 2617 (2021).
  133. J. Wildberger, M. Dax, S. R. Green, J. Gair, M. Pürrer, J. H. Macke, A. Buonanno, and B. Schölkopf, Phys. Rev. D 107, 084046 (2023).
  134. https://github.com/gwastro/ml-mock-data-challenge-1/blob/master/ds4.ini.

Outline

Information

Phys. Rev. D 112, 103002– Published 4 November, 2025
  • Received 27 January 2025
  • Accepted 3 October 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation