Export citation

Export citation

Choose format for download:

Download Citation

    GPU-accelerated subcycling time integration with the einstein toolkit

    Liwei Ji1,*, Roland Haas2,3,4, Yosef Zlochower1, Steven R. Brandt5, Erik Schnetter6,7,5, and Allen Wen1

    • 1Center for Computational Relativity and Gravitation, and School of Mathematical Sciences, Rochester Institute of Technology, 85 Lomb Memorial Drive, Rochester, New York 14623, USA
    • 2Department of Physics and Astronomy, University of British Columbia, Vancouver, Canada
    • 3National Center for Supercomputing applications, University of Illinois, 1205 West Clark Street, Urbana, Illinois, USA
    • 4Department of Physics, University of Illinois, 1110 West Green Street, Urbana, Illinois, USA
    • 5Center for Computation and Technology, Louisiana State University, Baton Rouge, Louisiana, USA
    • 6Perimeter Institute for Theoretical Physics, Waterloo, Ontario, Canada
    • 7Department of Physics and Astronomy, University of Waterloo, Waterloo, Ontario, Canada

    • *Contact author: ljsma@rit.edu

    Phys. Rev. D 112, 024049 – Published 21 July, 2025

    DOI: https://doi.org/10.1103/bqpq-5cp9

    Abstract

    Adaptive mesh refinement with subcycling in time enables different grid levels to advance using their own time steps, ensuring finer grids employ smaller steps for accuracy while coarser grids take larger steps to improve computational efficiency. We present the development, validation, and performance analysis of a subcycling in time algorithm implemented within the carpetx driver in the einstein toolkit framework. This new approach significantly improves upon the previous subcycling implementation in the carpet driver by achieving higher-order convergence—fourth order in time instead of second order—and enhanced scaling performance. The key innovation lies in optimizing the exchange of ghost points at refinement boundaries, limiting it to the same number as those at interprocess boundaries using dense output from coarser levels, thereby reducing computational and communication overhead compared to the implementation in carpet, which required a larger number of buffer zones. To validate the algorithm, we first demonstrate its fourth-order convergence using a scalar wave test. We then apply the algorithm to binary black hole simulations, confirming its robustness and accuracy in a realistic astrophysical scenario. The results show excellent agreement with the well-established lazev code. Scaling tests on CPU (Frontera) and GPU (Vista) clusters reveal significant performance gains, with the new implementation achieving improved speed and scalability compared to the carpet-based version.

    Physics Subject Headings (PhySH)

    Authorization Required

    We need you to provide your credentials before accessing this content.

    References (Subscription Required)

    Outline

    Information

    Sign In to Your Journals Account

    Filter

    Filter

    Article Lookup

    Enter a citation