I have a code that accumulates grad of each layer after .backward() call on loss.
It was working but after some change, I am seeing a model where all parameter with grad None
I guess since grad is None, no training is happening.
When does it usually happen? What should I check to find out the cause?
Couple notes about the model …
The model that I am having this issue is Discriminator of a GAN.
I generate a data from generator but call detach before I feed it to the discriminator.
also does using GPU make any difference in computation graph? t seems to be fine on CPU.