Yeah my use case is specifically keeping G frozen and generating samples from it - it is a test time application.
I will try this method, thanks!
Another thing -
After this loss.backward() call, I need to modify z_initial to clamp it like so:
z_initial = torch.clamp(z_initial, -1., 1.)

I am not able to do this and I am getting an error saying the code is trying to traverse the graph twice after some variables have been freed. I understand the error since it makes sense.
However, once I call the optimizer for z, I need to clamp it. Is there any other way this can be done?