Linux overcommits memory -- why?
Reading about the recently-discovered Apache security vulnerability, I learned that Linux uses a curious memory allocation strategy: it always allows `malloc` to succeed and lazily allocates the memory on first use instead. If not enough memory is available, it randomly selects a process in a somewhat ad-hoc manner (giving more weight to processes that are unprivileged, have tried to allocate lots of memory, etc) and kills it.

It seems to me that not only is it non-compliant behavior (in C at least; see http://stackoverflow.com/questions/1655650/linux-optimistic-malloc-will-new-always-throw-when-out-of-memory), and deprives programs the opportunity to gracefully handle low-memory conditions themselves, but it also doesn't stretch the imagination to come up with scenarios where killing an arbitrary process causes catastrophic data loss or corruption, or real-world accidents. Why is this design at all a good idea? Am I missing something? What were the design justifications?
Linux overcommits memory -- why?
===

Reading about the recently-discovered Apache security vulnerability, I learned that Linux uses a curious memory allocation strategy: it always allows `malloc` to succeed and lazily allocates the memory on first use instead. If not enough memory is available, it randomly selects a process in a somewhat ad-hoc manner (giving more weight to processes that are unprivileged, have tried to allocate lots of memory, etc) and kills it.

It seems to me that not only is it non-compliant behavior (in C at least; see http://stackoverflow.com/questions/1655650/linux-optimistic-malloc-will-new-always-throw-when-out-of-memory), and deprives programs the opportunity to gracefully handle low-memory conditions themselves, but it also doesn't stretch the imagination to come up with scenarios where killing an arbitrary process causes catastrophic data loss or corruption, or real-world accidents. Why is this design at all a good idea? Am I missing something? What were the design justifications?