Managing multiple communication methods in high-performance networked computing systems

Modern networked computing environments and applications often require - or can benefit from - the use of multiple communication substrates, transport mechanisms, and protocols, chosen according to where communication is directed, what is communicated, or when communication is performed. We propose techniques that allow multiple communication methods to be supported transparently in a single application, with either automatic or user-specified selection criteria guiding the methods used for each communication. We explain how communication link and remote service request mechanisms facilitate the specification and implementation of multimethod communication. These mechanisms have been implemented in the Nexus multithreaded runtime system, and we use this system to illustrate solutions to various problems that arise when multimethod communication is implemented. We also illustrate the application of our techniques by describing a multimethod, multithreaded implementation of the Message Passing Interface (MPI) standard, constructed by integrating Nexus with the Argonne MPICH library. Finally, we present the results of experimental studies that reveal performance characteristics of multimethod communication, the Nexus-based MPI implementation, and a large scientific application running in a heterogeneous networked environment.