You could profile your code using the profiler 3 or e.g. Nsight Systems 1 to check for bottlenecks and depending on these findings try to speed them up (e.g. the data loading in case your GPU would be waiting for the next data batches to arrive).