“log_alpha” is a variable(I think), every time I update its data with its grad, the grad will change into None.
code as follow:
1 alpha_loss = -torch.mean(self.log_alpha * (log_pis.double() + self._target_entropy))
2 #self.log_alpha.retain_grad()
3 alpha_loss.backward()
4 grad = self.log_alpha.grad
5 self.log_alpha = self.log_alpha - grad * 3e-4
but if I use line 2 code, no error. So I’m a little confused about the grad clearing mechanism.
& I’m not sure if the grad will be added all the time.