I’ve tried to run WGAN model and meet some strange behavior. I’m not sure is it my bug, or pytorch bug. I made a minimal example 14 to discuss it here.
So, we have an error

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

But if you look into the gist you can find my comments on how to change it to run successfully. For example, I don’t understand how inplace ReLu operations infuence on this error (note http://i.imgur.com/6H26H6o.png 3).
Main questions: can you explain why it works this way? Why those errors occur?
Details: pytorch 0.4.1, ubuntu 16.04, GeForce 1080TI
Note 1. Module BrokenBlock has no parameters. You can check it using the folowing code:
b = BrokenBlock(3)
for p in b.parameters():
    print(p.data.size())

And if it has no parameters then backward should not walk through it. But changing ReLu(inplace=False) in BrokenBlock somehow influences the backward to walk successfully.