Unleashing MPI: MorphMPI

Published on Thursday, 04 October 2007 20:00
Written by Toon Knapen
Hits: 11784

Why Simulate An ABI? Portability!



The MPI-standard specifies an API and not an ABI because, for instance, it does not define the layout of the MPI_Comm objects in memory nor does it define the exact value of MPI_ERR_COMM. The consequence of this design leaves plenty of flexibility (for optimization) to the MPI implementers.

Through striving for optimal performance, the MPI standard reduces portability, however. The MPI-standard forces applications to be launched using the same MPI-implementation as the one they were compiled against. This is no problem when the application is compiled and launched on the same machine. However this is a severe constraint for shrink-wrapped software.

What is a platform?
A platform-definition list the requirements on hard- and software-environment to be able to run an executable. These requirements mainly specify:
  • the processor (-family) necessary to run the executable because of the instruction-set used in the binary,
  • the operating-system because of the system-calls in the binary and
  • the interconnect for parallel (MPI) applications.
The dependency on the interconnect for MPI-applications is due to the lack of an MPI-ABI. This makes MPI-executables even less portable than their sequential counterparts.

Shrink-wrapped software is only available on a limited number of platforms which are selected by the application-developer. The HPC-world, in which MPI is mainly used, consists of many diverse platforms in contrast to the many X86 mainstream applications (e.g. office applications). Additionally, HPC applications are less portable compared to office-type applications: office applications do not care if they are being run on an Intel EM64T processor or AMD Opteron. HPC executables however, are generally not portable among these two processors because they would either exploit SSE3 or 3DNow instructions that are not supported by all processors. Because the application executables are linked to a specific MPI-library and run-time environment vendors find it difficult to provide shrink-wrapped HPC software optimized for the user's platform.

The advantages and disadvantages of standardizing an ABI for MPI have been discussed for many times. During one of those discussion on the beowulf-mailing list, a solution was put forward to implement a library on top of the existing MPI-implementations that would be able to provide binary compatibility [1] which Jeff Squyres coined MorphMPI. And because at Free Field Technologies [2] we regularly get requests to support yet another MPI-implementation, we finally decided to implement MorphMPI.

How Does It Work?

All problems in computer science can be solved by another level of indirection (D.Wheeler).

MorphMPI does exactly this. MorphMPI is a library on top of MPI. Applications using MorphMPI have no header-dependency on MPI and MorphMPI itself can be recompiled to use any MPI implementation. Thus object-code using MorphMPI can be linked with any MPI implementation.

How does it work?
Figure One: MorphMPI fits between the executable and MPI library (top). The traditioanl linking scheme is shown on the bottom

MorphMPI comes with a morphmpi.h header that contains all of the MPI symbols but prefixed with MorphMPI instead of MPI. The object-code of the application thus only contains references to MorphMPI-symbols. After linking the object-code to the MorphMPI-library and executing, the application will call MorphMPI-functions using MorphMPI-constants. At run-time, the morphmpi library will forward the MorphMPI-calls to actual MPI-calls while translating the MorphMPI-constants and passing on the corresponding MPI-objects. For instance, when the application calls {mosgoogle right}

  MorphMPI_Comm_size(MorphMPI_COMM_WORLD,p_size) ;
the MorphMPI_Comm_size function will first lookup the equivalent of the MorphMPI_COMM_WORLD communicator and than call MPI_Comm_size. In this case, a pre-defined communicator was used but a user-defined communicator could have been used as well. To be able to translate user-defined communicators, MorphMPI must map every MorphMPI-communicator to its equivalent MPI-communicator. This mapping is done in the communicator-constructor. For instance when calling
  MorphMPI_Comm_dup(MorphMPI_COMM_WORLD,p_my_comm) ;
the equivalent MPI-function will be called with MPI_COMM_WORLD as first argument and a pointer to a new variable of type MPI_Comm. After MPI_Comm_dup returns successfully, a new MorphMPI_Comm will be created and mapped to the newly created MPI_Comm. Thus, the next time MorphMPI_Send is called with this new MorphMPI-communicator, MorphMPI will be able to translate it to the equivalent MPI-communicator. Same goes for the datatype that is used in the MorphMPI_Send while the other arguments are forwarded to the MPI_Send function as is.

MorphMPI does not only provide an equivalent for the MPI C-interface but also the Fortran-interface. The Fortran interface is implemented in C and therefore relies on the C-interface of MorphMPI. This design ensures that MorphMPI should be able to translate the Fortran-specific constants such as MPI_COMPLEX in C. However not all MPI-implementations provide these constants in their C-interface.

Using MorphMPI








An Extra Feature





  1. beowulf mailing list: http://www.beowulf.org/archive/2005-March/012278.html
  2. FFT: http://www.fft.be
  3. ACTRAN: http://www.fft.be/?id=10
  4. MUMPS: http://graal.ens-lyon.fr/MUMPS/
  5. MorphMPI on sourceforge: http://morphmpi.sourceforge.net

Toon Knapen can be reached at toon.knapen ( a t ) telenet.be and would like to acknowledge Free Field Technologies for their assistance int his project.
