Ethernet Communication Layer for Open MPI


Ethernet Communication

Low-level Ethernet communication can help to decrease the latency and increase the bandwidth for the Ethernet network. This is done by circumvention of the traditional TCP/IP stack which was mainly developed for Wide Area Networks (WANs). However, cluster systems are mostly deployed in Local Area Networks (LANs) which makes many of TCP/IPs features superfluous. Optimized LAN Ethernet communication can decrease the CPU usage and latencies dramatically. We developed a portable Linux Kernel module and an Open MPI BTL to leverage this knowledge. The software was developed by (alphabetically) Sven Eckelmann, Torsten Hoefler, Mirko Reinhard, and Matthias Treydte.

The article [1] provides an overview of the software architecture.


Two master's theses by Mirko Reinhardt "Optimizing Point-to-Point Ethernet Cluster Communication" and Matthias Treydte Application Specific Optimization of Ethernet Cluster Communication continued implemented large parts of the project. I especially thank Sven Eckelmann for his efforts to fixing numerous bugs in the kernel part, porting the BTL to the latest Open MPI version, and increasing the code portability and readability significantly. Thanks a lot Sven!


Source Code

This project ended in 2008 (it was updated by Sven Eckelmann in Nov 2009) and the provided results and code are only here for reference and reproduceability. The code is not maintained or supported (use at own risk). If you are interested in continuing or maintaining this project, please contact me.


The code is available under a modified BSD license: mca_btl_eth.tgz - (86.7 kb) .

Wireshark Plugin for the ESP protocols

Sven Eckelmann created a plugin to dissect Ethernet Streaming Protocol (ESP) packets in wireshark. The plugin can be found at gitorious.

Related Projects

References

CIB-06-06
 [1] T. Hoefler, M. Reinhardt, F. Mietke, T. Mehlan, W. Rehm:
 Low Overhead Ethernet Communication for Open MPI on Linux Clusters TU Chemnitz. Vol CSR-06, Nr. 06, In Chemnitzer Informatik Berichte, presented in Chemnitz, TU Chemnitz, ISSN: 0947-5125, Jul. 2006,