Copyright Notice:

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

D. De Sensi, S. Di Girolamo, S. Ashkboos, S. Li, T. Hoefler:

 Flare: Flexible In-Network Allreduce

(In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC21), presented in St. Louis, Missouri, ACM, ISBN: , Nov. 2021, )

Abstract

The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation can be accelerated by offloading it to network switches, that aggregate the data received from the hosts, and send them back the aggregated result. However, existing solutions provide limited customization opportunities and might provide suboptimal performance when dealing with custom operators and data types, with sparse data, or when reproducibility of the aggregation is a concern. To deal with these problems, in this work we design a flexible programmable switch by using as a building block PsPIN, a RISC-V architecture implementing the sPIN programming model. We then design, model, and analyze different algorithms for executing the aggregation on this architecture, showing performance improvements compared to state-of-the-art approaches.

Documents

download article:
access preprint on arxiv:
download slides:


Recorded talk (best effort)

 

BibTeX

@inproceedings{flare,
  author={Daniele De Sensi and Salvatore Di Girolamo and Saleh Ashkboos and Shigang Li and Torsten Hoefler},
  title={{Flare: Flexible In-Network Allreduce}},
  year={2021},
  month={11},
  booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC21)},
  location={St. Louis, Missouri},
  publisher={ACM},
  isbn={},
  note={},
  doi={10.1145/3458817.3476178},
}