You could profile your code e.g. via Nsight Systems 4 or the PyTorch Profiler 5 to see where the bottleneck in your code is and to further isolate it. E.g. your training might suffer from a slow data loading and could explain the 50% utilization of the GPU.