Well, I don’t know, what kind of data you are dealing with, and how you are loading it.
Based on the type of data, you could probably tune the Python code even more.
Using multiple processes to load your data gives you the advantage, that the next batch might be ready waiting for the GPU to finish its current operation.
I’m not sure, how you are dealing with this in your C++ code, but if it’s sequential, you’ll have to wait for the data loading to finish, before you can push it onto the GPU, which might kill your speed advantage.
Also, with pin_memory=True you’ll save some unnecessary copying between pageable and pinned memory, when you would like to use the GPU.