Jim_Yang:

10 - 20 percent on Nvidia-smi and around 8% on task


may be try watch nvidia-smi then run the run the code in another terminal to see the usage realtime and I think there is always maximum speedup which you can achieve. Try increasing the batch size which will fit your GPU.