Yes, calling total_loss.backward() looks correct. I don’t know if the implemented training logic fits your use case, as I’m not familiar with it.


Yes, as long as no tensors are (accidentally) detached. To verify it, check the .grad attributes of the parameters of both models before and after the first backward() call. Before the call, they should be set to None, afterwards to a valid gradient tensor.


Yes and the same applies as in 2. If you are not detaching the tensors, the feature extractor would also get gradients.