Test environment: pytorch 0.4.1
Case 1:
1, torch.zeros([1,2], dtype=torch.float, requires_grad=True).is_leaf
True
2, torch.zeros([1,2], dtype=torch.float, requires_grad=True).cuda().is_leaf
False
3, torch.zeros([1,2], dtype=torch.float).cuda().is_leaf
True
Case 2:
y = torch.zeros([1,2]) # is_leaf = True
y = y.cuda() # is_leaf = True
y.requires_grad = True  # is_leaf = True
Seems that the 2 of Case 1 is not same with Case 2!!! Why?