A simple C++11 thread pool implementation
Threadpools are rabbit-hole problems.  They seem structurally simple at first, but they also have some unexpected failure modes.  There are two that I&#x27;m aware of, and there are almost certainly more.<p>1) Nonuniform work sizes.  Optimally scheduling the work when the sizes are nonuniform is Hard.  An almost-optimal algorithm is to dispatch larger items before smaller items.  This implementation dispatches work in FIFO order.  While this is Not Wrong (TM), its far from optimal when the work sizes are nonuniform.<p>2) Small work sizes.  As the size of each work item gets smaller and smaller, the proportion of the runtime cost which is spent to dispatch the work becomes larger and larger.  I&#x27;m working on a system which has dozens of work items to execute, but each one is only 50-100 microseconds long.  It turns out that this is an awfully inconvenient work size, and contention on a common FIFO was limiting available parallelism.  I found I got nearly 50% speedup by giving each worker thread its own workqueue and round-robin dispatching among them.<p>3) (edited to add) They don&#x27;t model fork&#x2F;join parallelism well on a common pool.  If work items also wait for work to complete, then the OP&#x27;s implementation may deadlock them.  To avoid deadlock, the join operator must be capable of executing a work item.  The C++11 future API isn&#x27;t rich enough to do this in user code.  See also WG21 paper N3557 for more on this topic.<p>From the perspective of a standards body, threadpools are similar to associative data structures.  There is a community of users who would like to have a one-size-fits-all implementation.  But it turns out that actual workloads have wildly differing requirements.  In practice, a standard library must provide several different models of a threadpool to satisfy them, or users will be forever complaining and&#x2F;or rolling their own.