Hi, vainaijr, I add them after loss.backward() and before optimizer.zero_grad(), layer connect with out2  have the grad, but without with out1 , I feel something wrong with the  part of out1, but I don’t know how to fix it.

%E6%8D%95%E8%8E%B71454×548 90.4 KB