Hi,
Thanks for the great link to the blog, I have read it and gained lots of insights about how autograd works but the article only explains the definition of leaf node but does not explain why it is needed, perhaps the author think it should be intuitive . It also does not answer some questions I had, such as:
>>> b = torch.rand(10, requires_grad=True).cuda()
>>> b.is_leaf
False
# b was created by the operation that cast a cpu Tensor into a cuda Tensor
>>> f = torch.rand(10, requires_grad=True, device="cuda")
>>> f.is_leaf
True
# f requires grad, has no operation creating it

Does the above mean that if we call cuda() on a tensor then we have to use retain_grad=True so that the grad attribute will be populated?