When Parallel: Pull, Don't Push
&gt; If we’re using concurrency, that means multiple threads of execution may be mutating the same cell, which requires synchronization — locks, atomics, etc. That much synchronization is the death knell of performance.<p>You can do this without synchronization by exploiting the fact that all &quot;lanes&quot; of a SIMD vector execute simultaneously, e.g. by doing vector scatters &quot;to the left&quot; and then &quot;to the right&quot; those writes will not race with each other, and don&#x27;t require locks or atomics.<p>Whether pull or push is faster heavily depends on the algorithm - there is no silver bullet here. To write efficient parallel algorithms you need to know how to do both.<p>The author only appears to know how to do pull (if that&#x27;s wrong, they didn&#x27;t really do push any justice in the article), so from their POV push just doesn&#x27;t work.<p>The article does show something very important though: pull does give you reasonable performance and is quite easy to reason about and get right. In contrast, push is quite hard due to the write-write data-races the author mentions, so I don&#x27;t blame the author from arriving at their conclusion that one should always do pull. It might not always be true, but it is definitely a safe place to start, particularly if you aren&#x27;t super familiar with the algorithm yet and don&#x27;t know how or if push would work well.<p>For example, many of the convolutions that neural networks use are actually implemented using the &quot;push&quot; approach because that&#x27;s more efficient, and they are quite similar to what the blog post author is doing.