Hi, I have been looking into the source code of the optimizer, zero_grad() function in particular.
    def zero_grad(self):
        r"""Clears the gradients of all optimized :class:`torch.Tensor` s."""
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is not None:
                    p.grad.detach_()
                    p.grad.zero_()

and I was wondering if one could just exchange
p.grad.detach_()
p.grad.zero_()

with
p.grad = None
In what cases these two options would make difference?