Performance modeling of a temperature accelerated molecular dynamics (AMD) code

-

-

(Left): Atomistic system used for the validation of TADSim, corresponding to an adatom on a silver (100) surface. The moving atoms are blue, the nonmoving atoms are black, and the adatom is shown in red. The adatom can hop to an adjacent binding site or perform a two-atom exchange to a next‐nearest-­ neighbor site. A large number of higher-­barrier processes are also available to this system. (Right): TADSim prediction of computational boost is shown as a function of THigh for different core‐counts for a silver system (TLow = 300K)

-

Next-generation high-performance computing (HPC) will require more scalable and flexible performance prediction tools to evaluate software–hardware co-design choices relevant to scientific applications and hardware architectures. Lab researchers examined a new class of tools called “application simulators”. These parameterized fast-running proxies of large-scale scientific applications use parallel discrete event simulation. Parameterized choices for the algorithmic method and hardware options provide a rich space for design exploration to determine well-performing software–hardware combinations. The team demonstrated the approach by modeling the temperature-accelerated dynamics (TAD) method, an algorithmically complex and parameter-rich member of the accelerated molecular dynamics (AMD) family of molecular dynamics methods. The work captures the essence of the TAD application as a TADSim simulator without the computational expense and resource usage of the full code.

A parameterized application simulator models the key stages of an application as discrete events while abstracting time-intensive parts or kernels of the application. The logic of an application or pseudocode is simulated, including loops, control flow, and termination conditions, similar to a state machine. Instrumentation is available for the collection of performance metrics. The software and hardware parameters specified in the simulator define the hardware and software design spaces. The application simulator approach is most useful for performance analysis of applications that do not have predictable progression prior to runtime. As high performance computing asynchronous programming models become more commonplace, parallel discrete event simulation performance prediction becomes an increasingly attractive choice. This approach is especially powerful in cases where the runtimes of certain sections of the codes are not constant but can vary significantly from one case to the next (e.g., if a nonlinear problem needs to be solved, or when the stop time of some procedure is a random variable).

The accelerated molecular dynamics method of temperature-accelerated dynamics illustrates this application simulation concept. TAD is an algorithm for reaching long time scales in molecular dynamics simulations. For most materials, the dynamical evolution on these longer time scales is characterized by infrequent events, in which the system makes occasional transitions from one state to another. An example is the jump of a vacancy in a solid or an atom on a surface. Much more complex events, sometimes involving many atoms, also occur. In a standard molecular dynamics simulation, this often means that considerable computing time can be invested without observing any significant microstructural change in the material. The AMD methods, of which TAD is one, exploit this infrequency characteristic to reach much longer times than would have been possible with molecular dynamics alone. In TAD, a system is advanced from state to state at a temperature TLow, where transitions are very rare, using information from simulations at a higher temperature THigh, where the dynamics proceeds much more rapidly. The essence of TAD is to statistically determine which of the transitions observed at THigh should have occurred first at TLow, in order to obtain proper dynamics. The process of advancing to a new state is stochastic, and follows from a number of different computational tasks (thermalization, running molecular dynamics, transition checks, etc.), each of which can take variable amounts of time to complete. Because of this algorithmic complexity, the number of parameters in the method, and the large number of ways the different steps can be carried out, predicting and optimizing the performance of TAD is challenging.

The team used TADSim to quickly characterize the runtime performance and algorithmic behavior for the otherwise long-running simulation code. Validation against the actual TAD code showed close agreement in force calls and computational boost for the evolution of an example physical system, a silver adatom (an extra isolated atom) on a silver (100) surface (Figure 2 left). Figure 2 (right) shows TADSim’s prediction of computational boost as a function of THigh for increasing core-count. The flexibility of the approach enabled the prediction of performance for new algorithm extensions, such as speculative spawning of the compute-bound stages, without having to implement such a method in the TAD codebase. Focused parameter scans allowed studies of algorithm parameter choices over far more scenarios than would have been possible with the actual simulation. This led to interesting performance-related insights and suggested extensions: 1) the optimal high temperature THigh decreases with increasing TLow, 2)spawning of certain tasks on different processors can be advantageous given a significant core-count budget,3) frequent transition checks improve performance in large core-count situations, but prove detrimental in limited-core situations, and 4) a high value for high temperature THigh increases the potential speedup, while a value that is too high degrades performance by introducing excessive overhead. These results add value for future method development and physics research in molecular dynamics.

Reference: “TADSim: Discrete Event-based Performance Prediction for Temperature Accelerated Dynamics, ACM Transactions on Modeling and Computer Simulation (TOMACS),” Volume 25, Issue 3, Article 15, April 2015; doi: 10.1145/2699715. http://dl.acm.org/citation.cfm?id=2699715Authors include: Susan Mniszewski and Stephan Eidenbenz (Information Sciences, CCS-3), Christoph Junghans (Applied Computer Science, CCS-7), Arthur Voter and Danny Perez (Physics and Chemistry of Materials, T-1).

Laboratory Directed Research and Development (LDRD) and a LANL Director’s postdoctoral fellowship sponsored the work, and the Institutional Computing Program at the Lab provided use of their high performance computing cluster resources. The research supported the Lab’s Energy Security mission area and the Information, Science, and Technology science pillars. Technical contacts: Susan Mniszewski and Arthur Voter