You can add this line:
advantages_minib = advantages_minib.detach()

after:
if self.normalize_adv:
    advantages_minib = (advantages_minib - advantages_minib.mean()) / (advantages_minib.std() + 1e-8)