I definitely agree that GPGPU has limitations in what it can do, but there&#x27;s a few details that are a bit skewed in your analysis. Perhaps your facts aren&#x27;t quite up to date (by, say, 5 years or so)<p>First: memory. A consumer GPU these days has 3 GiB of memory and a compute&#x2F;viz GPU has up to 12 GiB. The memory bandwidth is about 300 GiB&#x2F;s compared to 50 GiB per second for the CPU-DRAM. In other words: in memory intensive applications, the GPU wins. There&#x27;s no &quot;local memory&quot; in modern GPU architectures (unlike 5 or so years ago), they have L1&#x2F;L2 caches not unlike the CPU.<p>&gt; For all network and disk access it needs to communicate with the CPU, and again, this communication is slow.<p>This is certainly true that the GPU can&#x27;t do i&#x2F;o, but &quot;slow&quot; is an ambiguous term. Yes, it is high latency and difficult to orchestrate effectively but the memory bandwidth of a PCI-e bus is 16 GiB&#x2F;s (v3.x) to 32 GiB&#x2F;s (v4.x), that&#x27;s 30-60% of the bandwidth between the CPU and DRAM.<p>GPUs can handle complex in-memory data structures, and are very effective in processing e.g. tree structures. There&#x27;s even been applications of GPUs to databases (the PG-storm backend for postgres, for example).<p>Modern GPUs are entirely different beasts to what the matrix and vector batch processing chips were a few years ago were.<p>Yes, it&#x27;s a big shift in paradigms and requires some hard software engineering but the benefits are there. Case in point: the &quot;deep learning revolution&quot;, in particular training the networks, would not have happened were it not for GPU tech.