If i do this simply code:
  w=torch.tensor([1.0],requires_grad=True)
  w=w.cuda()
  #some compute 
  l.backward()
  #w.grad will still be NoneType

Then the tensor w will not be the leaf node in autograd’s graph (the w will be copybackward).
The parameters in net are all initialized as “requires_grad=True”.
So if i do net.cuda(),  is this equal to do param=param.cuda()? Could meet the same problem above?
Looking forward for apply!