A distributed system generally includes many loosely coupled computers, each of which typically include a computing resource (e.g., processor(s)) and storage resources (e.g., memory, flash memory, and/or disks). A distributed storage system overlays a storage abstraction (e.g., key/value store or file system) on the storage resources of a distributed system. In the distributed storage system, a server process running on one computer can export that computer's storage resources to client processes running on other computers. Remote procedure calls (RPC) may transfer data from server processes to client processes.
A remote procedure call is a two-sided software operation initiated by client software executing on a first machine and serviced by server software executing on a second machine. Servicing storage system requests (e.g., read data) in software may require an available processor, which may place a significant limitation on a distributed storage system. In the case of a distributed storage system, this means a client process cannot access a remote computer's storage resources unless the remote computer has an available processor to service the client's request. Moreover, the demand for processor resources and storage resources in a distributed system often do not match. In particular, computing resource (i.e., processors) may have heavy and/or unpredictable usage patterns, while storage resources may have light and very predictable usage patterns. When a server's processor(s) are heavily utilized, there may be no processors available to service a storage request when it arrives. In this case, the storage request waits for completion of other software tasks executed by the processor(s) or preemption by a kernel before the storage request can be serviced by a processor, even though there may be plenty of storage bandwidth to service the request immediately. Alternatively, one or more dedicated processors could service such storage requests, at the expense of efficiency and decreased processing capacity for nominal processing tasks.
Generally, coupling processor and storage resources can result in high and/or unpredictable latency, especially if the distributed system's processor resources are heavily utilized.