Could you use some random dummy data for training and check the GPU utilization?
If you see a higher GPU usage, you might have e.g. a data loading bottleneck, so that your GPU might be starving.
PS: I’m not a huge fan of setting the default tensor to a CUDATensor, as this creates an abstraction layer, where I’m not really sure, what’s going on under the hood.
If possible, create your model, data etc. and push them directly to the GPU via:
model = MyModel()
model.to('cuda')

for data, target in loader:
    data = data.to('cuda')
    target = target.to('cuda')
    output = model(data)

It’s a nitpick, but it makes the code much clearer in my opinion.