Copyright Notice:

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

T. Bonato, A. Rastegari, D. De Sensi, T. Schneider, S. Shen, A. Kubicek, P. Ram, T. Hoefler:

 Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance Sensitivity at Unprecedented Scale

(In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'26), presented in Chicago, IL, USA, IEEE, ISBN: 979-8-3195-4789-7, Nov. 2026)

Abstract

AI workloads on supercomputers make interconnect performance a primary design constraint. Hardware specifications and microbenchmarks alone do not show how real workloads respond to latency, bandwidth, and jitter. We present a trace-driven study of network sensitivity for training and inference workloads at scales up to 4,096 GPUs. Our methodology combines signature-based trace decomposition with linear programming to make broad network-parameter sweeps feasible. It preserves the recorded NCCL schedules and program dependencies, and estimates a lower bound on runtime under the LogGP model. We validate the compositional analysis against monolithic linear programs, LogGOPSim, and hardware measurements at feasible scales. The resulting latency-bandwidth envelopes identify low-sensitivity regions and performance cliffs that vary substantially across workloads. They also show when bandwidth upgrades provide meaningful returns and when additional network performance has little effect. These results translate execution traces into practical guidance for evaluating network configurations while making the model assumptions explicit.

Documents

download article:
access preprint on arxiv:
 

BibTeX

@inproceedings{bonato2026provisioning,
  author={Tommaso Bonato and Amirreza Rastegari and Daniele De Sensi and Timo Schneider and Siyuan Shen and Ales Kubicek and Prabhat Ram and Torsten Hoefler},
  title={{Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance Sensitivity at Unprecedented Scale}},
  year={2026},
  month={11},
  booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'26)},
  location={Chicago, IL, USA},
  publisher={IEEE},
  isbn={979-8-3195-4789-7},
}