As I presented in a previous article, there are 3 different models for the execution of MPI containers; the hybrid model being the most popular. A core concept for successfully using that model is to understand what combination of MPI implementations can be used on the host and in containers.
To help answer that question for the Arm64 architecture, I created such matrices on an Ubuntu Eoan testbed and Singularity 3.5.3. Now, my goal is not to show what combination always works but rather run experiments on a single node. If problems appear to use a single node, there is very little chance it could be successfully on multiple nodes.
To create the matrices, I used the validation tool developed by Sylabs (with a few minor modifications). This tool executes:
At the moment, it does not perform any tests for one-sided communications.
The tool is also set up to create all possible combinations for the following versions of MPICH: 3.0, 3.0.4, 3.1, 3.1.4, 3.2, 3.2.1, 3.3 and 3.3.2; and all the combination for the following versions of Open MPI: 3.0.4, 3.1.0, 3.1.4, 4.0.0, 4.0.1, 4.0.2. Finally, the tests are not set up to detect potential performance regressions; it only detects failures (pass or fail tests). The tool implicitly configures and installs MPICH and Open MPI. All tests were executed with Singularity 3.5.3.
We all need to be aware that the experiments only show a fraction of all the data that would be interesting to users. It is only the first piece of a global answer. The intent is to slowly gather more results by running more tests and analyzing more precisely the results.
All combinations of MPICH showed a successful execution of all tests. As previously stated, the tool does not detect potential performance regressions at the moment.
The same results are obtained with Open MPI.
This shows that the MPI developers do an excellent job at ensuring that different versions of their MPI implementations can talk to each other. For instance, in the context of Open MPI, it means that PMIx developers successfully enable cross-version communications.
These results are identical to results obtained on the x86_64 architecture. In other words, the MPI implementations on Arm64 seem to be as mature than for the x86_64 architecture, that is, the MPI and Singularity ecosystem on Arm64 is robust.
MPI containers execution models:
https://community.arm.com/developer/tools-software/hpc/b/hpc-blog/posts/mpi-containers-execution-models
SyValidate: https://github.com/sylabs/syvalidate
Singularity: https://github.com/sylabs/singularity
Open MPI: https://www.open-mpi.org
MPICH: http://www.mpich.org
PMIx: https://pmix.org