I’m sorry maybe i did not explain it clearly.  What I wanted to do is to re-create both the last fc layer and the fc layer within the auxiliary classifier, then just re-train the two layers. Therefore, for each epoch during training, we’ll have two outputs (one for auxiliary and one for the final fc layer) and two losses, loss_1 and loss_2, then do backprop by (loss_1 + loss_2).backward() .
Do you think it’ll work…？