Thanks for the flash reply!
In my example, net1 and net2 are generator and discriminator of a GAN and I think I can’t apply your suggestion as I am accumulating gradient of both networks over multiple mini batches due to low memory available. IN other to include your suggestion I would need to save gradient values every time I set the requires_grad flag to False and assign it back later … (please correct me if I’m wrong).
Does generator training need the forward pass of the discriminator to be tracked in the gradient flow? Is there a way to update the generator in a RL fashion? Just as if final_output was a sort of feedback from the environment (torch doesn’t throw any error, but gradients are none for both net1 and net2…).