I’m trying to implement the ConOpt optimizer 2 for GANs, which combines stochastic gradient descent for both the generator and discriminator with a gradient penalty regularizer. I know that standard practice is usually to give the generator and discriminator separate optimizers, but for this algorithm it is necessary to add the squared norm of the gradients from both models to each loss. Therefore, it is necessary to have something like the closure argument in the Optimizer.step() method, but somehow have two closures, one for each model. Does anyone have a suggested way of doing this with only one closure?