It should almost always be possible to get away with only two calls to .cuda(): one on the data and one on the model. Everywhere else you should use tensor.new to create tensors on the same device as other tensors you already have.