@albanD you are right in general and it is obvious. But I think you are wrong in the conclusions.
Consider the folowing changes:
        # gradient_penalty.backward()  # comment this line and error disappears
        # loss_D_real.backward()
        # loss_D_fake.backward()
        loss_D = loss_D_fake - loss_D_real + gradient_penalty
        loss_D.backward()

The error still occurs. It means that some vertices (nodes) of the graph were searched twice, but I don’t call .backward() twice as you said. So, I think that graph is backwarded twice by single call. It is obviously caused by autograd.grad routine but why error dessapears when Relu set not inplace?
On more thing. I haven’t found full description about retain_graph option and buffers it frees. What those buffers contain? Why I can or cann’t free them? Can you llink something for this topic to me?