• Home
• People
• Jobs
• Visit
• Thesis Topics
• Publications
• Conference Rankings
• Research
• Bcast
• Teaching
• Contact

SPONSORS

COLLABORATIONS

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

R. L. Castro, A. Ivanov, D. Andrade, T. Ben-Nun, B. B. Fraguela, T. Hoefler:
	VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores (In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23), presented in Denver, CO, USA, Association for Computing Machinery, ISBN: 979-8-400701-09-2, Nov. 2023) Publisher Reference Abstract The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2x speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37x speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers. Documents download article: access preprint on arxiv: download slides: Recorded talk (best effort)
	BibTeX
@inproceedings{,   author={Roberto L. Castro and Andrei Ivanov and Diego Andrade and Tal Ben-Nun and Basilio B. Fraguela and Torsten Hoefler},   title={{VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores}},   year={2023},   month={11},   booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23)},   location={Denver, CO, USA},   publisher={Association for Computing Machinery},   isbn={979-8-400701-09-2},   doi={10.1145/3581784.3607099}, }