Thanks for your reply. But my understanding of how optimizer works is that they take .grad  and current weight and then update them. So even through discriminator’s optimizer does not get called, .grad gets calculated for discriminator’s weight when you call .backward and on the forward pass, cache is saved as require_grad==True. Let me know if that is the case or I mis-understand something.