Engineering.
production freeze, we were seeing an unusually high activity of such rollouts or tests. These activities created, updated or deleted pods at such a high rate (order of thousands of pods in a matter of minutes) that this caused Kubernetes API server’s watch cache to go stale quickly. In essence,