10 Million Concurrent Connections – The Kernel is the Problem
> The kernel is the problemI had an epiphany one day when I realized that the kernel is nothing but a library with an expensive calling convention.The only reason we bother calling the kernel at all is because it has privileges that userspace programs don't have, and it uses those privileges to control/mediate access to shared resources.The downside of the kernel is that there's no way around it.  You can't "opt out" of any of its decisions.  With any normal library, if had an O(n^2) algorithm somewhere, or wasn't optimized for your use case, or just generally got in the way, you could choose not to link against it.  User-space libraries are democratic in this respect; you vote with your linker line.  But with the kernel, it's "my way or the highway."  The kernel is the only way to the hardware.Here's an unfortunate example: O_DIRECT is one of those few ways that you can sidestep the kernel.  With O_DIRECT you can bypass the kernel's page cache, which there are very good reasons for wanting to do.  But Linus's opinion is that "I should have fought back harder": https://lkml.org/lkml/2007/1/10/233  He thinks it's unfortunate that you can sidestep the kernel, because "You need a buffer whatever IO you do, and it might as well be the page cache."But what if you want better memory accounting, or better isolation between users, or a different data structure, or any number of other tweaks to the page cache implementation?  Well thankfully we have O_DIRECT.  Otherwise, your only choice would be to try to convince the Linux kernel maintainers to integrate your change, after you've tweaked it so that it's suitable for absolutely everyone else that uses Linux, and given it an interface that Linux is willing to support forever.  Good luck with that.The kernel has always been the problem.  User-space is where it's at.