Prefix sum on portable compute shaders
&gt; there is something slightly taboo about coordination between workgroups in the same dispatch. Many GPU experts I’ve talked with express skepticism that this can work at all. Even so, interest in this type of pattern is picking up, in part because of advanced rendering engines like Nanite, which also uses atomics to coordinate work between workgroups in a live-running dispatch.<p>I live only in CUDA land, I’m a bit ignorant of the other platforms, so I’m not sure what granularity of workgroups is here. Generally speaking you can’t rely on individual threads executing in any particular order, no matter how many there are, so yes you can’t have one thread look at data from another thread without some kind of synchronization. It took me longer than it should have to really grok this and to believe it deeply, initially I kept wanting to think that thread number 235 million could read from thread 0 safely without synchronizing because surely enough time has passed. Nope, threads can and do come in any order, it’s never safe to assume another thread has finished.<p>Using atomics to solve this is rarely a good idea, atomics will make things go slowly, and there is often a way to restructure the problem so that you can let threads read data from a previous dispatch, and break your pipeline into more dispatches if necessary.<p>CUDA at least has a few other ways to share data during a single dispatch (or “launch”) besides atomics. Threads in warps can talk to each other, threads in blocks can share memory. But this all takes careful design and various other synchronization primitives.<p>&gt; That is sadly not the case for GPU compute code. The most common scenario is dependency on a large, vendor-dependent toolkit such as CUDA (that installer is a 2.4GB download). If you have the right hardware, and the runtime installed properly, then your code can run.<p>While GPU coding is indeed more onerous than CPU programming in general, I feel like this wasn’t necessarily a fair point - this is the CUDA SDK download being compared to the CPU runtime. Installing the Rust compiler &amp; cargo wasn’t mentioned as a downside, for example. CPU code also requires the right hardware and to have the runtime installed properly, it’s just something most people already have setup. Similarly, compiled CUDA code will run just fine without the SDK, and most people attempting to run a compiled CUDA program will have a compatible driver installed already. For the average compiled app, the run-time is no trickier than CPU, and the CPU has more or less the same kinds of requirements.