Hi,
I’m training an autoencoder using this set of scripts 6 (specifically the attention-based parts) with cuda.
I’ve enabled cuda on every tensor I can find, and also set the usual suspects:
model.cuda()
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.fastest = True

It’s definitely enabled and I can see a bit of GPU memory in use, but there is no performance speed-up at all. In fact (probably due to all the additional initialisation) I’m seeing a marginal decrease in performance.
Am I missing something obvious, or are the tensor operations simply not particularly suited to GPU?
Note: functionally the scripts work exactly as expected and produce a working trained model.
Thanks!