Distributed Lock Management with RDMA: Decentralization without Starvation
Lock managers are a crucial component of modern distributed systems. However, with the increasing availability of fast RDMA enabled networks, traditional lock managers can no longer keep up with the latency and throughput requirements of modern systems. Centralized lock managers can ensure fairness and prevent starvation using global knowledge of the system, but are themselves single points of contention and failure. Consequently, they fall short in leveraging the full potential of RDMA networks. On the other hand, decentralized (RDMA based) lock managers either completely sacrifice global knowledge to achieve higher throughput at the risk of starvation and higher tail latencies, or they resort to costly communications in order to maintain global knowledge, which can result in significantly lower throughput. In this paper, we show that it is possible for a lock manager to be fully decentralized and yet exchange the partial knowledge necessary for preventing starvation and thereby reducing tail latencies. Our main observation is that we can design a lock manager primarily using RDMA's fetch and add (FA) operations, which always succeed, rather than compare and swap (CAS) operations, which only succeed if a given condition is satisfied. While this requires us to rethink the locking mechanism from the ground up, it enables us to sidestep the performance drawbacks of the previous CAS based proposals that relied solely on blind retries upon lock conflicts. Specifically, we present DSLR (Decentralized and Starvation free Lock management with RDMA) a decentralized lock manager that targets distributed systems running on RDMA enabled networks. We demonstrate that, despite being fully decentralized, DSLR prevents starvation and blind retries by guaranteeing first come first serve (FCFS) scheduling without maintaining explicit queues. We adapt Lamport's bakery algorithm to an RDMA enabled environment with multiple bakers, utilizing only one sided READ and atomic FA operations. Our experiments show that, on average, DSLR delivers 1.8x (and up to 2.8x) higher throughput than all existing RDMA based lock managers, while reducing their mean and 99.9% latencies by 2.0x and 18.3x (and up to 2.5x and 47x) respectively.
SIGMOD Conference