• Home
• People
• Jobs
• Visit
• Thesis Topics
• Publications
• Conference Rankings
• Research
• Bcast
• Teaching
• Contact

SPONSORS

COLLABORATIONS

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

T. Hoefler, T. Bonato, D. De Sensi, S. Di Girolamo, S. Li, M. Heddes, J. Belk, D. Goel, M. Castro, S. Scott:
	HammingMesh: A Network Topology for Large-Scale Deep Learning (In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'22), Nov. 2022) SC22 Reproducibility Advancement Award and Invited as CACM Research Highlight Abstract Numerous microarchitectural optimizations unlocked tremendous processing power for deep neural networks that in turn fueled the AI revolution. With the exhaustion of such optimizations, the growth of modern AI is now gated by the performance of training systems, especially their data movement. Instead of focusing on single accelerators, we investigate data-movement characteristics of large-scale training at full system scale. Based on our workload analysis, we design HammingMesh, a novel network topology that provides high bandwidth at low cost with high job scheduling flexibility. Specifically, HammingMesh can support full bandwidth and isolation to deep learning training jobs with two dimensions of parallelism. Furthermore, it also supports high global bandwidth for generic traffic. Thus, HammingMesh will power future large-scale deep learning systems with extreme bandwidth requirements. Documents download article: access preprint on arxiv: Recorded talk (best effort)
	BibTeX
@inproceedings{,   author={Torsten Hoefler and Tommaso Bonato and Daniele De Sensi and Salvatore Di Girolamo and Shigang Li and Marco Heddes and Jon Belk and Deepak Goel and Miguel Castro,Steve Scott},   title={{HammingMesh: A Network Topology for Large-Scale Deep Learning}},   year={2022},   month={11},   booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'22)}, }