Cache made consistent: Meta’s cache invalidation solution
I see a couple of monitoring&#x2F;reporting systems.. but no caching solution. These are tools to catch bugs in the solution you&#x27;re using. Good work, but not a solution for cache invalidation.<p>And regarding those tools:<p>it doesnt sound like Polaris would handle a network partition well. If a cache invalidation triggers it to check the other caches for consistency... that assumes Polaris will receive that invalidation message. Imagine a scenario of 5 cache servers, and a Polaris server. On one side of the split is Polaris and 2 servers, and the other has 3 cache servers... It&#x27;s possible for the 3 cache servers to receive an update that is not received by the polaris+2 network... And polaris would not just be unaware of the inconsistency, but it also wouldn&#x27;t know to check later for the inconsistency when the network partition is resolved.<p>I also feel like the consistency tracing is assuming that only one fill and invalidate is occurring at a time (when in practice, there may be multiple fills and invalidates occurring in parallel on the same data)... and that those calls will arrive in order. If they arrive out of order, it doesnt sound like it would catch that.. and I think you&#x27;re relying on Polaris to catch this case, but high latency cannot be differentiated from a network partition except the length of the delay... so these two types of errors can be seen together.. in which case, you&#x27;d have a cache error that neither tool would detect.<p>I would like to hear why Im wrong.<p>I understand this is being used in prod, but network partitions don&#x27;t occur everyday... and Im not convinced this has seen enough flaky networks to work out the bugs.