Your proposed solution (read from disk on startup and never again) is really a memory-backed data store, not a cache. Caches can miss.<p>But let&#x27;s analyze your example. If disk reads take tens of seconds and memory usage is high enough to purge the kernel&#x27;s disk cache, nothing can save you. Had your process read in everything at the start, it would be using even more memory. Given the same load, one of two things will happen:<p>1. If you have swap enabled, parts of your process&#x27;s memory will be swapped-out. Accessing &quot;memory&quot; in this case would cause a page fault and tens of seconds of delay.<p>2. If you have swap disabled, the OOM-killer will reap your process. When it respawns, it&#x27;s going to read lots of stuff from disk... and disk reads take tens of seconds. Oops.<p>Even if an application-level data cache improved performance on heavily-loaded shared hosts, the added costs of software development and maintenance far exceed the cost of better hardware. Hardware is cheap. Developers are expensive.