Hi,
Thanks, I should maybe have clarified, I’m already running other scripts successfully using cuda on this instance. For example there is a set of RNNs for which I’m seeing a very clear 15X speed-up when enabling cuda. Although the attention-based autoencoder works a little differently, I was hopeful of seeing at least a measurable increase when enabling cuda.
What I’m trying to understand is why there isn’t any improvement in performance. For example, are the type of tensor operations in the attention-based autoencoder not optimised for GPU? Or am I missing something more obvious?
I don’t want to clutter the thread with my version of the scripts, but basically I perform an initial check with:
torch.cuda.is_available()
This succeeds as True, and subsequent enablings of cuda on tensors proceed without error.
Regarding your question, I’ve tried the attention-based scripts (with some modifications for cuda) on two different Cloud platforms. One of them is an AWS Sagemaker instance, info below.
Instance type: ml.p2.xlarge (4 X vCPU, 1 X K80 GPU, 61GiB Memory)
Notebook kernel: conda_pytorch_p27
Kernel: 4.14.72-68.55.amzn1.x86_64 (Amazon modified)
OS: Red Hat 7.2.1-2
Pytorch: 0.4.1
Cuda: 9.2.148
Cudnn: 7104
These are all provided by Sagemaker by default.
Thanks!