Hi!
Sorry I am coming definitely late, but I’m facing your identical problem. I have implemented the actor loss as:
pred_a1 = self.actor.forward(s1)
actor_Loss = -1*torch.mean(self.critic.forward(s1, pred_a1))

but this makes the actor diverge.
I do not how to fix that problem… Did you solve it?
Thank you a lot in advance!