A Method to Reduce Load Imbalances in Simulations of Solidification Processes with Free Surface 3D

The present work introduces a first step towards reducing load imbalances in simulations of phase change processes with Free Surface 3D (FS3D). FS3D is a program for the Direct Numerical Simulation (DNS) of multiphase flows. It is able to simulate complex deformations of interfaces between phases by means of a Volume-of-Fluid (VOF) method. The work is focused on the model for the phase transition process from supercooled water to hexagonal ice. In order to investigate complex phenomena with a high computational cost FS3D uses the computing power of the supercomputer Cray XC-40 at the High Performance Computing Center (HLRS). During the calculations, the computational costs for elements that contain both the solid and the fluid phase is higher than for elements which contain only one of the phases. If the computational domain is decomposed into equal parts, the workload is inhomogeneously distributed among the cores. The presented method is able to distribute the workload more homogeneously among the cores and, therefore, enables an efficient use of the computational resources. Elements with higher computational costs are identified by the Volume-of-Fluid (VOF) method. Consequently these elements are associated with a higher computational load in form of a weight. This information is passed to a recursive bisection algorithm which performs the domain decomposition. The recursive bisection of the computational domain considers the existing data structure of FS3D and provides contiguous arrays. To realize the process communication, a nearest neighbour communication was implemented with non-blocking Message Passing Interface (MPI) routines. The diagonal elements are transported via a communication sequence in order to avoid communication of small amounts of data which minimizes the communication overhead.