Deep inside the Linux kernel: a network latency spike
> Setting the rmem sysctl to only 2MiB is not recommended as it could affect the performance of high throughput, high latency connectionsSo what I took away from this article is that they'd set some absurdly high receive buffer, which predictably caused something to break, and never quite got around to explaining how they arrived at the original number. That buffer is for data the kernel has received but has not yet been read() by userspace. Seriously, 5MB of RAM dedicated to each socket in the kernel?Assuming these are Nginx proxy connections, why did you choose to tweak a kernel tunable rather than increasing the corresponding userspace application buffer sizes instead?edit: so the answer seems to be that Nginx is wholly reliant on the kernel to handle buffering, which is completely asinine to me, but perhaps there is good rationale here I've somehow missed. The take home becomes "our app stack is so broken we managed to break additional extreme-battle-tested components in a bid to to get it to function sensibly"