Solved by ptrblck in post #2 
              
              
                You could create two different forward passes through NetB with the output of NetA and with the detached output of NetA: 
outA = NetA(input)
outB = NetB(outA)
outB_det = NetB(outA.detach())
outC = NetC(outB)

loss = criterion(outC, target)
loss.backward() # will calculate the grads for A, B, C

loss…