Thanks a lot. I pulled those lines out of GAN model, and move them to later training phase and it seems to be working. 
for param in self.discriminator.parameters():
            param.requires_grad = False