&gt; I&#x27;d suggest creating a templated buffer class that manages data uploads&#x2F;downloads and exposes the appropriate pointer on the appropriate device.<p>This is the easy part.<p>The hard part is getting this right in the big picture, to avoid having the CPU wait for the GPU and vice versa. Pipeline stalls will easily ruin your performance, to the point where single threaded CPU code runs faster because the high latency of data transfers.<p>This is where modern GPU memory management really helps. For example, in OpenGL, you can use GL_MAP_BUFFER_PERSISTENT and _COHERENT to map your buffers once right after you create them and keep the pointer to the mapped buffer for the entire lifetime of the buffer (see ARB_buffer_storage). When you do this, you&#x27;re responsible for the synchronization so that the CPU and GPU never touch the same parts of the buffer concurrently. CUDA probably has something similar.<p>This adds some complexity to the application but now it&#x27;s done explicitly in the application and there are no surprises. And the CPU&#x2F;GPU page tables don&#x27;t have to be modified, TLBs flushed and so on, so you get a bit of extra perf from there.<p>And I totally agree with the point about the mindset of single threaded programmers with dynamic languages. I was once put into a project where a scientific application was written in Python and some &quot;big matrix&quot; operations were done in C. The matrix ops were on the top of the profile so they assumed that it can be made faster by simply &quot;porting&quot; them over to GPGPU.<p>Well the matrices weren&#x27;t that big (a few kilobytes) and the operations were done sequentially, one by one in the application. To make it work, the operations would have had to be batched but this would have meant a big refactoring to the entire Python codebase. This was way too complex for the chemists and phycisists who had written the code. They were really smart people with the domain knowledge but they didn&#x27;t possess the mindset to think about making it efficient.