In the classifier tutorial listed here: http://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html 2
There is a line in the loop that has this:
running_loss += loss.data[0]
This line seems to be about 30% slower than doing running_loss += loss.data when running_loss is initialized as a float tensor, and I believe this is because running_loss = 0 sits on the CPU and loss sits on the GPU. Is there any downside to setting:
running_loss = torch.cuda.FloatTensor(np.zeros(1))
And having the line changed as shown above?