Solved by Ali1234 in post #2 
              
              
                I had similar case. When I checked the loss value of inner model’s output after each epoch, it was different (decreasing). This means that the model weights are changing which implies the model is training. So, as far as you don’t use detach() the model2 is training. To verify, you can also return t…