Copyright Notice:

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

T. Hoefler, M. Reinhardt, F. Mietke, T. Mehlan, W. Rehm:

 Low Overhead Ethernet Communication for Open MPI on Linux Clusters

(Vol CSR-06, Nr. 06, In Chemnitzer Informatik Berichte, presented in Chemnitz, TU Chemnitz, ISSN: 0947-5125, Jul. 2006)


This paper describes the basic concepts of our solution to improve the performance of Ethernet Communication on a Linux Cluster environment by introducing Reliable Low Latency Ethernet Sockets. We show that about 25% of the socket latency can be saved by using our simplified protocol. Especially, we put emphasis on demonstrating that this performance benefit is able to speed up the MPI level communication. Therefore we have developed a new BTL component for Open MPI, an open source MPI-2 implementation which offers with its Modular Component Architecture a nearly ideal environment to implement our changes. Microbenchmarks of MPI collective and Point-to-Point operations were performed. We see a performance improvement of 8% to 16% for LU and SP implementations of the NAS parallel benchmark suite which spends a significant amount of time in the MPI. Practical application tests with Abinit, an electronic structure calculation program, show that the runtime of be nearly halved on a 4 node system. Thus we show evidence that our new Ethernet communication protocol is able to increase the speedup of parallel applications considerably.




  author={Torsten Hoefler and M. Reinhardt and Frank Mietke and Torsten Mehlan and Wolfgang Rehm},
  title={{Low Overhead Ethernet Communication for Open MPI on Linux Clusters}},
  institution={TU Chemnitz},
  booktitle={Chemnitzer Informatik Berichte},
  publisher={TU Chemnitz},