Special issue paper reviews CoPA particle application advancements

-

-

Anatomy of a time step is shown for each of the particle application submotifs addressed by the Exascale Computing Project’s Co-design Center for Particle Applications. Communication-intensive steps and compute/memory-intensive steps are shown in yellow and blue, respectively.

-

The Exascale Computing Project’s (ECP’s) Co-design Center for Particle Applications (CoPA) aims to prepare particle applications for exascale computing. A recent article in a special issue paper of the International Journal of High Performance Computing Applications by CoPA researchers, with Laboratory researcher Susan M. Mniszewski the lead author, among other Lab contributors and researchers from other laboratories and universities, reviews particle application developments at the Center.

CoPA provides proxy applications (apps) and libraries that enable exascale readiness. CoPA’s research focuses on submotifs that address (1) short-range particle–particle interactions, which often dominate molecular dynamics (MD) and smoothed particle hydrodynamics methods; (2) long-range particle–particle interactions, used in electrostatic MD and gravitational N-body methods; (3) particle-in-cell methods; and (4) linear-scaling electronic structure and quantum molecular dynamics (QMD) algorithms.

Proxy apps such as CoPA-developed ExaMiniMD, CabanaMD, CabanaPIC, and ExaSP2 are used to evaluate the viability of incorporating various algorithms, data structures, and architecture-specific optimizations and associated trade-offs. CoPA’s two main libraries—Cabana Particle Toolkit and PROGRESS/BML QMD Libraries—optimize data structure, layout, and movement to deliver performance portability, flexibility, and scalability across architectures with and without GPU acceleration. The Cabana Particle Toolkit applies to short-ranged, long-ranged, and particle-grid interactions, while PROGRESS/BML QMD Libraries provide methods and solvers for quantum mechanical interactions. The libraries are used by ECP and non-ECP application partners. CoPA includes members from the following ECP partner applications: WDMapp/XGC (Princeton Plasma Physics Laboratory), EXAALT/LAMMPS/LATTE (Sandia National Laboratory, LANL), ExaSky/HACC (Argonne National Laboratory), and ExaAM/PicassoMPM (Oak Ridge National Laboratory).

CoPA projects have shown that proxy apps are beneficial for rapid prototyping of different ideas and performance speedup and that co-design teams of domain scientists, computational scientists, and expert programmers in hardware-specific languages and programming models is an effective work mode for advancing particle applications. Specific project impacts include the transitioning of XGC from FORTRAN to C++ using the Cabana Particle Toolkit and Kokkos and the development of a hybrid quantum mechanical/molecular mechanical QM/MM protein simulation capability, which has proven useful in biomedical research including studies of SARS-CoV-2 proteins.

CoPA’s ongoing work on particle application readiness for exascale computing includes using Cabana/Kokkos in XGC for GPU off-loading of the ion particle operation in the ITER tokamak for performance-portable simulations on exascale platforms; improving additive manufacturing simulations with the PicassoMPM code; and developing HACC-based proxy apps towards a potential full N-body cosmological simulation code.

Mission and Funding

This work was supported by the Department of Energy, and performed as part of the CoPA, supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of the DOE, Office of Science and the NNSA. The work supports the Global Security mission area and the Information Science and Technology capability pillar.

Reference

“Enabling particle applications for exascale computing platforms,” The International Journal of High Performance Computing Applications, 0, 0, 1-26 (2021); DOI: 10.1177/10943420211022829. Authors: Susan M. Mniszewski, Michael E. Wall (CCS-3); Adetokunbo A. Adedoyin, Christoph Junghans, Jamaludin Mohd-Yusof, Robert F. Bird (CCS-7); Guangye Chen (T-5); Christian F. A. Negre (T-1); Shane Fogerty (XCP-2); Salman Habib, Adrian Pope (ANL); James Belak, Daniel Osei-Kuffor, Lee Ricketson (LLNL); Jean-Luc Fattebert, Stuart R. Slattery, Damien Lebrun-Grandie, Samuel Temple Reeve (ORNL); Choongseok Chang, Stephane Either, Amil Y. Sharma (PPPL); Steven J. Plimpton, Stan G. Moore (SNL); Aaron Scheinberg (Jubilee Development).

Technical Contact: Susan Mniszewskiv