Sure!
If you don’t detach fake_data_on_gpu the fake_loss.backward() call will compute the gradients in the discriminator and the generator, since the computation graph was created as
random input -> generator -> discriminator -> loss function
In the discriminator update step you don’t need the gradients in the generator for the “fake” label, so you could save computing these gradients by detaching the generator output as:
random input -> generator -> detach output -> discriminator -> loss function
Now the backward call will only calculate all gradients to the point the graph was detached.
In the generator update step, you are using a “real” label and need the gradients in the generator, so you have to use the first approach without detaching the graph.
That being said, you will not only save some unnecessary computation (and maybe even wrong gradients if you don’t clear them), but you might also get an error as in your case.
After the fake_loss.backward() call, intermediate activations in the models will be cleared to save memory. Since you need these intermediates for the generator_loss.backward() call, the error was raised.
Let me know, if something is still unclear.