Hi @ptrblck , I’m also having the same questions about how to only train part of network, and thanks for the answer!
One follow-up question is, what is the best-practice of gating in multi-task learning?
The context is that,

here in OP’s question there are 2 tasks/losses, and gating is desired during training.
but from OP’s code both tasks are computed for every training example, which is a waste of cpu, because we only want only loss not both.

So should we use an if-else branching inside forward to only activate 1 task? Or what is the best-practice?