Mathematically, it is not correct, because by doing fix grad, you got only part of gradients for theta_A
But physically, it is, because B is supposed to be a well trained network, and B(A(x)) is supposed to have a better performance than A(x). If you don’t do the fix, the other part of the gradients will try to make the B(A(x)) to be like A(x), and it is exactly not what we want.