Reading NFS at 25GB/s using FIO and libnfs
I&#x27;ve tried to drive NFS to reasonable levels of performance in the past, and the bottleneck has never seemed like storage or network or the NFS server; it always seems like the combination of the built-in NFS client in the Linux kernel, the implementation of filesystem semantics, and the behavior of common workloads ends up making NFS <i>much</i> slower than network throughput. I&#x27;m impressed with these benchmarks and the approach to collecting them, but I&#x27;m wondering if the Linux kernel NFS client can get anywhere close to the theoretical limits.<p>I&#x27;ve tried this with a <i>read-only</i> NFS server, across an AWS multi-gigabit connection, and I still found that I couldn&#x27;t get anywhere near this level of performance for the workload of &quot;make -j$(nproc)&quot; in a Linux kernel tree. As a quick baseline, some numbers from the last time I tested this, with a c5.12xlarge (48 CPU) client and server: a defconfig local build was 40s, and a defconfig build with a Linux kernel tree in read-only NFS (with a tmpfs overlay on top for writability) was 6m55s. That&#x27;s a 10x slowdown. System stats during the build showed 4-5MBps net recv and net send, and 1-4MBps disk write.<p>Is there some well-known method to getting reasonable performance out of off-the-shelf NFS servers and clients?