Copyright Notice:

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Publications of SPCL

T. Hoefler:

 Towards scalable RDMA locking on a NIC

(Presentation - presented in Palo Alto, CA, USA, Aug. 2016, )


We discuss how low-level functionality such as locking should move into the network. We start with the design of a topology-aware distributed Reader-Writer lock that accelerates irregular workloads for supercomputers and data centers. The core idea behind the lock is a modular design that is an interplay of three distributed data structures: a counter of readers/writers in the critical section, a set of queues for ordering writers waiting for the lock, and a tree that binds all the queues and synchronizes writers with readers. Each structure is associated with a parameter for favoring either readers or writers, enabling adjustable performance that can be viewed as a point in a three dimensional parameter space. We also develop a distributed topology-aware MCS lock that is a building block of the above design and improves state-of-the-art MPI implementations. Both schemes use non-blocking Remote Memory Access (RMA) techniques for highest performance and scalability. We evaluate our schemes on a Cray XC30 and illustrate that they outperform state-of-the-art MPI-3 RMA locking protocols by 81% and 73%, respectively. Finally, we use them to accelerate a distributed hashtable that represents irregular workloads such as key-value stores or graph processing. We then describe an abstract machine model for offload-enabled architectures. The Portals 4 network interface is used to implement the proposed abstract model. We show how complex network algorithms such as collective operations can be offloaded with this machine model. We believe that other services such as locks can be offloaded as well.


download slides:


  author={Torsten Hoefler},
  title={{Towards scalable RDMA locking on a NIC}},
  location={Palo Alto, CA, USA},