I have seen the topics discussed about this error,
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

but I think in my case is different. I have implemented a PPO algorithm where the actor and the critic are two completely different networks, and so I backward the actor loss for the actor network, and the critic loss for the critic network, as shown below:
# critic loss backward implementation
self.critic_optimizer.zero_grad()
critic_loss.backward()
nn.utils.clip_grad_norm_(self.critic.parameters(), 0.5)
self.critic_optimizer.step()

# actor loss backward implementation
self.actor_optimizer.zero_grad()
actor_loss.backward() ######### ERROR ARISES HERE
nn.utils.clip_grad_norm_(self.actor.parameters(), 0.5)
self.actor_optimizer.step()

The thing is that although these networks are different, the error occurs when I call the backward of the actor loss (the second backward call in my code), which is not related to the backward of the critic loss. I there a way for pytorch to understand that the actor’s backward is not related with the critic’s backward?