Hi,
I am implementing an A2C algorithm, the loss function for the critic is simply the advantage function and that works.
For the actor I use this:
loss = -distribution.log_prob(sample)*advantage.detach()
loss.requires_grad = True
loss.backwards()

where advantage is calculated using the critic and sample is from sampling from the distribution I am trying to learn.
variance_energy = 0.01
variance_decision = 0.01
mean_matrix = torch.FloatTensor([mean_energy, mean_decision])
covariance_matrix = torch.FloatTensor([[variance_energy, 0],[0, variance_decision]])
distribution = MultivariateNormal(mean_matrix, covariance_matrix)
sample = distribution.sample()

Here, mean_energy and mean_decision are the output of my NN and are restricted to positive values (so I don’t have problems with the log)
The gradient is always None.
I have already checked that required_grad = True and is_leaf = True for the loss function also.
I calculate the gradients of the weights manually (so I can update them that way) after calling loss.backward()  with:
 for p in self.model.parameters():
            gradient_list.append(p.grad)

Does  anybody have an idea how to troubleshoot this? Thanks in advance!