• Home
• People
• Jobs
• Visit
• Thesis Topics
• Publications
• Conference Rankings
• Research
• Bcast
• Teaching
• Contact

SPONSORS

COLLABORATIONS

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

M. Besta, P. Renc, R. Gerstenberger, P. Sylos Labini, A. Ziogas, T. Chen, L. Gianinazzi, F. Scheidl, K. Szenes, A. Carigiet, P. Iff, G. Kwasniewski, R. Kanakagiri, C. Ge, S. Jaeger, J. Wąs, F. Vella, T. Hoefler:
	High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations (In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23), presented in Denver, CO, USA, Association for Computing Machinery, ISBN: 979-8-400701-09-2, Nov. 2023) Publisher Reference Abstract Graph attention models (A-GNNs), a type of Graph Neural Networks (GNNs), have been shown to be more powerful than simpler convolutional GNNs (C-GNNs). However, A-GNNs are more complex to program and difficult to scale. To address this, we develop a novel mathematical formulation, based on tensors that group all the feature vectors, targeting both training and inference of A-GNNs. The formulation enables straightforward adoption of communication-minimizing routines, it fosters optimizations such as vectorization, and it enables seamless integration with established linear algebra DSLs or libraries such as GraphBLAS. Our implementation uses a data redistribution scheme explicitly developed for sparse-dense tensor operations used heavily in GNNs, and fusing optimizations that further minimize memory usage and communication cost. We ensure theoretical asymptotic reductions in communicated data compared to the established message-passing GNN paradigm. Finally, we provide excellent scalability and speedups of even 4–5x over modern libraries such as Deep Graph Library. Documents download article: download slides:
	BibTeX
@inproceedings{besta2023gnn,   author={Maciej Besta and Paweł Renc and Robert Gerstenberger and Paolo Sylos Labini and Alexandros Ziogas and Tiancheng Chen and Lukas Gianinazzi and Florian Scheidl and Kalman Szenes and Armon Carigiet and Patrick Iff and Grzegorz Kwasniewski and Raghavendra Kanakagiri and Chio Ge and Sammy Jaeger and Jarosław Wąs and Flavio Vella and Torsten Hoefler},   title={{High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations}},   year={2023},   month={11},   booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23)},   location={Denver, CO, USA},   publisher={Association for Computing Machinery},   isbn={979-8-400701-09-2},   doi={10.1145/3581784.3607067}, }