The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.
Publications of SPCL
|T. Hoefler, P. Gottschling, A. Lumsdaine:
|Brief Announcement: Leveraging Non-blocking Collective Communication in High-performance Applications
(In Proceedings of the Twentieth Annual Symposium on Parallelism in Algorithms and Architectures, SPAA'08, presented in Munich, Germany, pages 113-115, Association for Computing Machinery (ACM), ISSN: , ISBN: 978-1-59593-973-9, Jun. 2008, (short paper) )
AbstractAlthough overlapping communication with computation is an important mechanism for achieving high performance in parallel programs, developing applications that actually achieve good overlap can be difficult. Existing approaches are typically based on manual or compiler-based transformations. This paper presents a pattern and library-based approach to optimizing collective communication in parallel high-performance applications, based on using non-blocking collective operations to enable overlapping of communication and computation. Common communication and computation patterns in iterative SPMD computations are used to motivate the transformations we present. Our approach provides the programmer with the capability to separately optimize communication and computation in an application, while automating the interaction between computation and communication to achieve maximum overlap. Performance results results with two model applications show more than 90% decrease in communication overhead, resulting in 16% and 21% overall performance improvements.