Hi Harpal!



 harpalsahota:

The training process looks like this:
        meta_cats_loss = Variable(bce_loss(meta_preds.detach().cpu(), meta_cats_batch), requires_grad=True)

        loss = mbmr_loss + meta_cats_loss
        loss.backward()

The issue lies with the meta_cats_loss.


This detach() is wrong:
bce_loss(meta_preds.detach().cpu(), meta_cats_batch)

It “breaks the computation graph” in that it  “detaches” meta_preds
from the computation of bce_loss() so that you don’t  backpropagate
through, and optimize, the weights that were used to produce
meta_preds.
That is, meta_cats_loss has no effect on your training.
(Note, Variable has been deprecated since a long time ago, so you
don’t need or want it.  Also, the requires_grad=True in your Variable
doesn’t undo the damage done by detach().)
I haven’t looked at the rest of your code, but this issue is a good place
to start.
Best.
K. Frank