I see.
Well, either you save the log_prob of the action with its computation graph, or you recalculate the log_prob from the state before you calculate the losses.
So, either you accept ballooning memory, or you accept redoing computations.