Surprising Economics of Load-Balanced Systems
This can be flipped into a simple heuristic known as the &quot;square root staffing rule&quot; (sometimes &quot;... law&quot;). The name comes from call centres.<p>Basically, the number of servers you need to serve X amount of demand with Y probability of queuing is not linear with X. It is proportional to the square root of X.<p>The intuition is that a call centre agent is either talking to a customer, or they are not. If they are not talking to a customer, then they can immediately serve a customer. The odds that a given agent is talking to a customer is on a probability distribution, usually assumed to be the normal distribution.<p>In a normal distribution, larger samples lead to more and more appearances of outliers in the sample. More agents means more &quot;outliers&quot;, in this case, more idle agents. And that curve is not <i>linear</i>, it follows the shape of the normal distribution.<p>Where it sucks for call centres is that demand shows seasonality and shift planning has relatively inflexible lead times. But in a software scenario we can typically acquire additional capacity quickly, so precision in forecasts is less of a problem.<p>A decent explanation: <a href="https:&#x2F;&#x2F;www.networkpages.nl&#x2F;the-golden-rule-of-staffing-in-contact-centers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.networkpages.nl&#x2F;the-golden-rule-of-staffing-in-c...</a><p>A short scenario: <a href="https:&#x2F;&#x2F;www.xaprb.com&#x2F;blog&#x2F;square-root-staffing-law&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.xaprb.com&#x2F;blog&#x2F;square-root-staffing-law&#x2F;</a><p>A simple calculator linked from the short scenario: <a href="https:&#x2F;&#x2F;www.desmos.com&#x2F;calculator&#x2F;8lazp6txab" rel="nofollow">https:&#x2F;&#x2F;www.desmos.com&#x2F;calculator&#x2F;8lazp6txab</a>