So the issue is that the p3 has insufficient compute to match the gpu - you need this compute for loading images from disk, and for transforming each image. Some solutions - change the loading format to something that loads faster - like hdf5 . I haven’t tested this but I know that loading is a huge bottle neck. You might also notice that the problem goes away if yo use a larger model- basically the gpu becomes the bottleneck as desired. What we’re really after tho is cost effective training and not being wasteful- which in some cases the g3 might be a better option since it has a higher ratio of compute to gpu. You can also experiment with using less transforms, or trying to implement transforms that would normally be implemented by the cpu in the model itself so that it’s done by the gpu. There also might be opportunities to improve the efficiency of transforms. Lastly, setting num workers to 8 helped me