Copyright Notice:

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

B. Arimilli, R. Arimilli, V. Chung, S. Clark, W. Denzel, B. Drerup, T. Hoefler, J. Joyner, J. Lewis, J. Li, N. Ni, R. Rajamony:

 The PERCS High-Performance Interconnect

(Vol , Nr. , In Proceedings of 18th Symposium on High-Performance Interconnects (Hot Interconnects 2010), presented in , pages , IEEE, ISSN: , ISBN: , Aug. 2010, )

Publisher Reference

Abstract

The PERCS system was designed by IBM in response to a DARPA challenge that called for a high-productivity high-performance computing system. A major innovation in the PERCS design is the network that is built using Hub chips that are integrated into the compute nodes. Each Hub chip is about 580 mm2 in size, has over 3700 signal I/Os, and is packaged in a module that also contains LGA-attached optical electronic devices. The Hub module implements five types of high-bandwidth interconnects with multiple links that are fully-connected with a high-performance internal crossbar switch. These links provide over 9 Tbits/second of raw bandwidth and are used to construct a two-level direct-connect topology spanning up to tens of thousands of POWER7 chips with high bisection bandwidth and low latency. The Blue Waters System, which is being constructed at NCSA, is an exemplar large-scale PERCS installation. Blue Waters is expected to deliver sustained Petascale performance over a wide range of applications. The Hub chip supports several high-performance computing protocols (e.g., MPI, RDMA, IP) and also provides a non-coherent system-wide global address space. Collective communication operations such as barriers, reductions, and multi-cast are supported directly in hardware. Multiple routing modes including deterministic as well as hardware-directed random routing are also supported. Finally, the Hub module is capable of operating in the presence of many types of hardware faults and gracefully degrades performance in the presence of lane failures.

Documents

download article:
download slides:
 

BibTeX

@inproceedings{ibm-percs-network,
  author={B. Arimilli and R. Arimilli and V. Chung and S. Clark and W. Denzel and B. Drerup and Torsten Hoefler and J. Joyner and J. Lewis and J. Li and N. Ni and R. Rajamony},
  title={{The PERCS High-Performance Interconnect}},
  institution={IBM},
  year={2010},
  month={08},
  pages={},
  volume={},
  number={},
  booktitle={Proceedings of 18th Symposium on High-Performance Interconnects (Hot Interconnects 2010)},
  location={},
  publisher={IEEE},
  issn={},
  isbn={},
  note={},
}