Ah, I figured it out! The problem is that I’m using the pytorch-lightning package, and it was freezing the weights of the other model for each step. On the discriminator step, the gradient norm is actually fully a function of the generator weights, so that was causing the problem. So it looks like I’ll have to probably write a custom optimizer to get the functionality I want.