Thanks for your prompt reply, Soumith!
I mean what if the two are not called simultaneously, maybe in two functions.
Is it still possible to do that?
Would lossAB.backward(retain_variables=True) helps? However if it is, it is not a good option since it retains all variables in netA and netB.