Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations--颗粒学报PARTICUOLOGY

Vaidhynathan, D., Sitaraman, H., Grout, R., Hauser, T., Hrenya, C. M., & Musser, J. (2023). Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations. Particuology, 78, 97-110. https://doi.org/10.1016/j.partic.2022.10.004

Memory access optimization for particle operations in computational fluid dynamics-discrete element method simulations

Deepthi Vaidhynathan ^a *, Hariswaran Sitaraman ^a, Ray Grout ^a, Thomas Hauser ^b, Christine M. Hrenya ^c, Jordan Musser ^d

a Computational Science Center, National Renewable Energy Laboratory, United States
b Research Computing, University of Colorado, Boulder, United States
c Chemical and Biological Engineering, University of Colorado, Boulder, United States
d Computational Science and Engineering, National Energy Technology Laboratory, United States

10.1016/j.partic.2022.10.004

Volume 78, July 2023, Pages 97-110

Received 27 June 2022, Revised 30 September 2022, Accepted 16 October 2022, Available online 8 November 2022, Version of Record 2 January 2023.

E-mail: Deepthi.Vaidhynathan@nrel.gov

Highlights

Abstract

Computational Fluid Dynamics – Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. These optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ∼ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

Graphical abstract

Keywords

文章导读