That’s interesting, I do think here retain_graph=False doesn’t work even if create_graph=True with normal discriminator loss i.e. gradient_penalty - d_real_out + d_fake_out
First of all, as per Pytorch docs, retain_graph=None defaults to create_graph.
So when we give following:
... create_graph=True, retain_graph=False,...

and use above test code with slight changes to only one line:
gp = ((gradients.norm(dim=1)-1)**2).mean() + y.mean()

it fails with:
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

So it does seem like retain_graph=True (or None if create_graph=True) is necessary for normal discriminator loss.