- Physical Review Journals
All JournalsPhysics Magazine

Physical Review D

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Reinforcement learning and metaheuristics for Feynman-integral reduction

Mao Zeng*

  • Higgs Centre for Theoretical Physics, School of Physics and Astronomy, University of Edinburgh, Edinburgh EH9 3FD, United Kingdom

  • *Contact author: mao.zeng@ed.ac.uk
PDF

Phys. Rev. D 112, 114051 – Published 31 December, 2025

DOI: https://doi.org/10.1103/dmlf-jkfc

Abstract

We propose new methods for optimizing the integration-by-parts (IBP) reduction of Feynman integrals, an important computational bottleneck in modern perturbative calculations in quantum field theory. Using the simple example of one-loop massive bubble integrals, we pose the problem of minimizing the number of arithmetic operations in reducing a target integral to master integrals via the Laporta algorithm. This is a nontrivial combinatorial optimization problem over the ordering of IBP equation generation (from pairs of seed integrals and IBP operators) and the ordering of integral elimination. Our first proposed method is reinforcement learning, which involves an agent interacting with an environment in a step-by-step manner and learning the best actions to take given an observation of the environment (in this case, the current state of the IBP reduction process). The second method is using metaheuristics, e.g., simulated annealing, to minimize the computational cost as a black-box function of numerical priority values that control the orderings. For large-scale problems, the number of free parameters can be compressed by using a small neural network to assign priority values. Remarkably, with almost no human guidance, both methods lead to IBP reduction schemes that are competitive with the most efficient human-designed algorithms. We also found interpretable features in the AI results that may be applicable to more complicated problems.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (76)

  1. K. G. Chetyrkin and F. V. Tkachov, Integration by parts: The algorithm to calculate β-functions in 4 loops, Nucl. Phys. B192, 159 (1981).
  2. S. Laporta, High-precision calculation of multiloop Feynman integrals by difference equations, Int. J. Mod. Phys. A 15, 5087 (2000).
  3. Charalampos Anastasiou and Achilleas Lazopoulos, Automatic integral reduction for higher order perturbative calculations, J. High Energy Phys. 07 (2004) 046.
  4. C. Studerus, Reduze-Feynman integral reduction in C++, Comput. Phys. Commun. 181, 1293 (2010).
  5. Roman N. Lee, LiteRed 1.4: A powerful tool for reduction of multiloop integrals, J. Phys. Conf. Ser. 523, 012059 (2014).
  6. A. V. Smirnov, Algorithm FIRE—Feynman Integral REduction, J. High Energy Phys. 10 (2008) 107.
  7. A. V. Smirnov and V. A. Smirnov, FIRE4, LiteRed and accompanying tools to solve integration by parts relations, Comput. Phys. Commun. 184, 2820 (2013).
  8. Alexander V. Smirnov, FIRE5: A C++ implementation of Feynman Integral REduction, Comput. Phys. Commun. 189, 182 (2015).
  9. A. V. Smirnov and F. S. Chukharev, FIRE6: Feynman Integral REduction with modular arithmetic, Comput. Phys. Commun. 247, 106877 (2020).
  10. Alexander V. Smirnov and Mao Zeng, FIRE 6.5: Feynman integral reduction with new simplification library, Comput. Phys. Commun. 302, 109261 (2024).
  11. Philipp Maierhöfer, Johann Usovitsch, and Peter Uwer, Kira—A Feynman integral reduction program, Comput. Phys. Commun. 230, 99 (2018).
  12. Philipp Maierhöfer and Johann Usovitsch, Kira 1.2 release notes, arXiv:1812.01491.
  13. Jonas Klappert, Fabian Lange, Philipp Maierhöfer, and Johann Usovitsch, Integral reduction with Kira 2.0 and finite field methods, Comput. Phys. Commun. 266, 108024 (2021).
  14. Fabian Lange, Johann Usovitsch, and Zihao Wu, Kira 3: Integral reduction with efficient seeding and optimized equation selection, arXiv:2505.20197.
  15. R. N. Lee, Presenting LiteRed: A tool for the Loop InTEgrals REDuction, arXiv:1212.2685.
  16. O. V. Tarasov, Computation of Grobner bases for two loop propagator type integrals, Nucl. Instrum. Methods Phys. Res., Sect. A 534, 293 (2004).
  17. Vladimir P. Gerdt and Daniel Robertz, A Maple package for computing Grobner bases for linear recurrence relations, Nucl. Instrum. Methods Phys. Res., Sect. A 559, 215 (2006).
  18. A. V. Smirnov and Vladimir A. Smirnov, Applying Grobner bases to solve reduction problems for Feynman integrals, J. High Energy Phys. 01 (2006) 001.
  19. A. V. Smirnov, An Algorithm to construct Grobner bases for solving integration by parts relations, J. High Energy Phys. 04 (2006) 026.
  20. A. V. Smirnov and V. A. Smirnov, S-bases as a tool to solve reduction problems for Feynman integrals, Nucl. Phys. B, Proc. Suppl. 160, 80 (2006).
  21. R. N. Lee, Group structure of the integration-by-part identities and its application to the reduction of multiloop integrals, J. High Energy Phys. 07 (2008) 031.
  22. Mohamed Barakat, Robin Brüser, Claus Fieker, Tobias Huber, and Jan Piclum, Feynman integral reduction using Gröbner bases, J. High Energy Phys. 05 (2023) 168.
  23. Janusz Gluza, Krzysztof Kajda, and David A. Kosower, Towards a basis for planar two-loop integrals, Phys. Rev. D 83, 045012 (2011).
  24. Robert M. Schabinger, A new algorithm for the generation of unitarity-compatible integration by parts relations, J. High Energy Phys. 01 (2012) 077.
  25. Harald Ita, Two-loop integrand decomposition into master integrals and surface terms, Phys. Rev. D 94, 116015 (2016).
  26. Kasper J. Larsen and Yang Zhang, Integration-by-parts reductions from unitarity cuts and algebraic geometry, Phys. Rev. D 93, 041701(R) (2016).
  27. S. Abreu, F. F. Cordero, H. Ita, M. Jaquier, B. Page, and M. Zeng, Two-loop four-gluon amplitudes from numerical unitarity, Phys. Rev. Lett. 119, 142001 (2017).
  28. Samuel Abreu, Fernando Febres Cordero, Harald Ita, Ben Page, and Mao Zeng, Planar two-loop five-gluon amplitudes from numerical unitarity, Phys. Rev. D 97, 116014 (2018).
  29. Janko Böhm, Alessandro Georgoudis, Kasper J. Larsen, Hans Schönemann, and Yang Zhang, Complete integration-by-parts reductions of the non-planar hexagon-box via module intersections, J. High Energy Phys. 09 (2018) 024.
  30. Dominik Bendle, Janko Böhm, Wolfram Decker, Alessandro Georgoudis, Franz-Josef Pfreundt, Mirko Rahn, Pascal Wasser, and Yang Zhang, Integration-by-parts reductions of Feynman integrals using Singular and GPI-Space, J. High Energy Phys. 02 (2020) 079.
  31. Zihao Wu, Janko Boehm, Rourou Ma, Hefeng Xu, and Yang Zhang, NeatIBP 1.0, a package generating small-size integration-by-parts relations for Feynman integrals, Comput. Phys. Commun. 295, 108999 (2024).
  32. Zihao Wu, Janko Böhm, Rourou Ma, Johann Usovitsch, Yingxuan Xu, and Yang Zhang, Performing integration-by-parts reductions using NeatIBP 1.1 + Kira, Comput. Phys. Commun. 316, 109798 (2025).
  33. Pierpaolo Mastrolia and Sebastian Mizera, Feynman integrals and intersection theory, J. High Energy Phys. 02 (2019) 139.
  34. Hjalte Frellesvig, Federico Gasparotto, Stefano Laporta, Manoj K. Mandal, Pierpaolo Mastrolia, Luca Mattiazzi, and Sebastian Mizera, Decomposition of Feynman integrals by multivariate intersection numbers, J. High Energy Phys. 03 (2021) 027.
  35. Xiao Liu and Yan-Qing Ma, Determining arbitrary Feynman integrals by vacuum integrals, Phys. Rev. D 99, 071501(R) (2019).
  36. Xin Guan, Xiao Liu, and Yan-Qing Ma, Complete reduction of integrals in two-loop five-light-parton scattering amplitudes, Chin. Phys. C 44, 093106 (2020).
  37. David A. Kosower, Direct solution of integration-by-parts systems, Phys. Rev. D 98, 025008 (2018).
  38. Bo Feng, Chang Hu, Jiyuan Shen, and Yaobo Zhang, General one-loop generating function by IBP relations, Phys. Rev. D 111, 076026 (2025).
  39. Philipp Kant, Finding linear dependencies in integration-by-parts equations: A Monte Carlo approach, Comput. Phys. Commun. 185, 1473 (2014).
  40. Andreas von Manteuffel and Robert M. Schabinger, A novel approach to integration by parts reduction, Phys. Lett. B 744, 101 (2015).
  41. Tiziano Peraro, Scattering amplitudes over finite fields and multivariate functional reconstruction, J. High Energy Phys. 12 (2016) 030.
  42. S. Abreu, J. Dormans, F. F. Cordero, H. Ita, and B. Page, Analytic form of planar two-loop five-gluon scattering amplitudes in QCD, Phys. Rev. Lett. 122, 082002 (2019).
  43. Jonas Klappert and Fabian Lange, Reconstructing rational functions with FireFly, Comput. Phys. Commun. 247, 106951 (2020).
  44. Tiziano Peraro, FiniteFlow: Multivariate functional reconstruction using finite fields and dataflow graphs, J. High Energy Phys. 07 (2019) 031.
  45. Giuseppe Laurentis and Daniel Maître, Extracting analytical one-loop amplitudes from numerical evaluations, J. High Energy Phys. 07 (2019) 123.
  46. Jonas Klappert, Sven Yannick Klein, and Fabian Lange, Interpolation of dense and sparse rational functions and other improvements in FireFly, Comput. Phys. Commun. 264, 107968 (2021).
  47. Giuseppe De Laurentis and Ben Page, Ansätze for scattering amplitudes from p-adic numbers and algebraic geometry, J. High Energy Phys. 12 (2022) 140.
  48. Vitaly Magerya, Rational tracer: A tool for faster rational function reconstruction, arXiv:2211.03572.
  49. A. V. Belitsky, A. V. Smirnov, and R. V. Yakovlev, Balancing act: Multivariate rational reconstruction for IBP, Nucl. Phys. B993, 116253 (2023).
  50. Herschel A. Chawdhry, p-adic reconstruction of rational functions in multiloop amplitudes, Phys. Rev. D 110, 056028 (2024).
  51. Xiao Liu, Reconstruction of rational functions made simple, Phys. Lett. B 850, 138491 (2024).
  52. Andreas Maier, Scaling up to multivariate rational function reconstruction, Comput. Phys. Commun. 317, 109827 (2025).
  53. T. Gehrmann and E. Remiddi, Differential equations for two-loop four-point functions, Nucl. Phys. B580, 485 (2000).
  54. Mathias Driesse, Gustav Uhre Jakobsen, Gustav Mogull, Jan Plefka, Benjamin Sauer, and Johann Usovitsch, Conservative black hole scattering at fifth post-Minkowskian and first self-force order, Phys. Rev. Lett. 132, 241402 (2024).
  55. Xin Guan, Xiao Liu, Yan-Qing Ma, and Wen-Hao Wu, Blade: A package for block-triangular form improved Feynman integrals decomposition, Comput. Phys. Commun. 310, 109538 (2025).
  56. Zvi Bern, Enrico Herrmann, Radu Roiban, Michael S. Ruf, Alexander V. Smirnov, Vladimir A. Smirnov, and Mao Zeng, Amplitudes, supersymmetric black hole scattering at O(G5), and loop integration, J. High Energy Phys. 10 (2024) 023.
  57. Aurélien Dersy, Matthew D. Schwartz, and Xiaoyuan Zhang, Simplifying polylogarithms with machine learning, Int. J. Data Sci. Math. Sci. 1, 135 (2024).
  58. Ryusuke Jinno, Gregor Kälin, Zhengwen Liu, and Henrique Rubira, Machine learning post-Minkowskian integrals, J. High Energy Phys. 07 (2023) 181.
  59. Francesco Calisto, Ryan Moodie, and Simone Zoia, Learning Feynman integrals from differential equations with neural networks, J. High Energy Phys. 07 (2024) 124.
  60. Tianji Cai, Garrett W. Merz, François Charton, Niklas Nolte, Matthias Wilhelm, Kyle Cranmer, and Lance J. Dixon, Transforming the bootstrap: Using transformers to compute scattering amplitudes in planar N=4 super Yang–Mills theory, Mach. Learn. Sci. Tech. 5, 035073 (2024).
  61. Clifford Cheung, Aurélien Dersy, and Matthew D. Schwartz, Learning the simplicity of scattering amplitudes, SciPost Phys. 18, 040 (2025).
  62. Matt von Hippel and Matthias Wilhelm, Refining integration-by-parts reduction of Feynman integrals with machine learning, J. High Energy Phys. 05 (2025) 185.
  63. Zhuo-Yang Song, Tong-Zhi Yang, Qing-Hong Cao, Ming-xing Luo, and Hua Xing Zhu, Explainable AI-assisted Optimization for Feynman integral reduction, arXiv:2502.09544.
  64. Stefan Weinzierl, Feynman Integrals. A Comprehensive Treatment for Students and Researchers, UNITEXT for Physics (Springer, Cham, 2022).
  65. Andrew G. Barto and Richard S. Sutton, Reinforcement Learning: An Introduction (MIT Press, 1998).
  66. Andrei Constantin, Thomas R. Harvey, and Andre Lukas, Heterotic string model building with monad bundles and reinforcement learning, Fortschr. Phys. 70, 2100186 (2022).
  67. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
  68. See Supplemental Material at http://link.aps.org/supplemental/10.1103/dmlf-jkfc for implementation details and lengthy data.
  69. Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah, Julia: A fresh approach to numerical computing, SIAM Rev. 59, 65 (2017).
  70. Anthony Corso and Robert Moss, Crux.jl, GitHub (2025), https://github.com/sisl/Crux.jl.
  71. Emile H. L. Aarts and Peter J. M. Van Laarhoven, Simulated annealing: A pedestrian review of the theory and some applications, in Pattern Recognition Theory and Applications (Springer, New York, 1987), pp. 179–192.
  72. Jesús-Adolfo Mejía de Dios and Efrén Mezura-Montes, Metaheuristics: A Julia package for single- and multi-objective optimization, J. Open Source Software 7, 4723 (2022).
  73. Héctor Corte, Simulated annealing optimization, MATLAB Central File Exchange (2011) (retrieved: April 21, 2025).
  74. Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulao, Andreas Kallinteris, Markus Krimmel, Arjun KG et al., Gymnasium: A standard interface for reinforcement learning environments, arXiv:2407.17032.
  75. Simon Danisch and Julius Krumbiegel, makie.jl: Flexible high-performance data visualization for Julia, J. Open Source Software 6, 3349 (2021).
  76. Alexander LeNail, NN-SVG: Publication-ready neural network architecture schematics, J. Open Source Software 4, 747 (2019).

Outline

Information

Phys. Rev. D 112, 114051– Published 31 December, 2025
  • Received 23 June 2025
  • Accepted 5 December 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI. Funded by SCOAP3.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation