because you have clearly called inputs.cuda() and labels.cuda() it is definitely using the GPU.
I suspect you are bottlenecked on data loading. Try increasing the num_workers in your DataLoader and see if that helps.