The code where we call .cuda() on the variable:
a = Variable(torch.randn(2,10), requires_grad=True).cuda()
y = a + 10.0
y.backward(torch.ones(a.size()).cuda())
print a.grad

The output is:
Variable containing:
0     0     0     0     0     0     0     0     0     0
0     0     0     0     0     0     0     0     0     0
[torch.cuda.FloatTensor of size 2x10 (GPU 0)]

which is incorrect.
If we modify it and call .cuda() on the tensor:
a = Variable(torch.randn(2,10).cuda(), requires_grad=True)
y = a + 10.0
y.backward(torch.ones(a.size()).cuda())
print a.grad

The output is correct:
        Variable containing:
        1     1     1     1     1     1     1     1     1     1
        1     1     1     1     1     1     1     1     1     1
    [torch.cuda.FloatTensor of size 2x10 (GPU 0)]

I think this rather strange behavior needs some clarification or an exception throwing.
My pytorch version is 0.1.9+49295eb