Depending on what pytorch version you are on Variable() is actually long deprecated and I am not sure you need non_blocking to be True for your application, but all in all your code should work.
You might wanna check if you have other tensors on the GPU. If you do anything with the tensor that is your models output, like logging, you might wanna check if it gets freed correctly, like I mentioned here:



 RaLo4:

Check if you are maybe putting different tensors onto the GPU and keeping them there.
If you for example do total_loss += loss instead of total_loss += loss.item() for logging your loss, your GPU memory will fill up.