Hi Ryan!



 machina:

I am trying to change the params of gen() such that a cross-entropy loss: f_loss = CE_loss(f_pred, labels) is maximized and a different type of loss: lossG = torch.mean(torch.log(1. - output)) is minimized.
f_pred = target(adv_ex) #.size() = [32, 10]
f_loss = CE_loss(f_pred, labels) #add loss for gens desired f pred
print(f_loss)
loss_G_Final = f_loss+lossG # can change the weight of this loss term later
     
opt_gen.zero_grad()
loss_G_Final = loss_G_Final.to(device)
loss_G_Final.backward()
opt_gen.step()



I haven’t looked at you code in detail, and I’m not sure I understand
the signs of your losses.
Let me say what I think you’re trying to do.
First, when a model is doing “better,” its associated loss is lower.
Pytorch’s optimizers, such a Adam, seek to minimize the loss.
I assume that f_loss is measuring how well your generator, gen,
is generating a “fake” sample with certain desired characteristics.
If a lower value of f_loss means that gen is doing better, then
calling f_loss.backward() and opt_gen.step() will move gen's
parameters in the direction that improves gen's performance (as
measured by f_loss).
Now I assume that lossG measures how good your discriminator,
disc, is at telling the “fake” samples generated by gen from real
samples.  Smaller values of lossG mean that your discriminator
is working better.
The complexity comes because you want to train disc to do well,
that is to lower lossG.  But you want to train gen so that disc
has a hard time distinguishing fake from real, that is to increase
lossG
Assuming that disc's parameters are (temporarily) frozen, you want
to train gen to lower f_loss while increasing lossG.  Doing this is as
simple as training gen with (note the minus sign):
loss_G_Final = f_loss - lossG

(You may well choose to weight f_loss and lossG differently to
adjust gen's trade-off in producing the characteristics measured
by f_loss vs. tricking disc, as measured by lossG.)
One final note:  When you train disc to successfully distinguish
fake from real, you will not want to flip the sign of lossG.  You
would call:
lossG.backward()
opt_disc.step()

f_loss doesn’t depend on disc's parameters (unless your code is
hiding something somewhere), so you neither need nor want to call
f_loss.backward() (nor loss_G_Final.backward()) before calling
opt_disc.step().
Best.
K. Frank