The background of this question is as same as this one. I realized that to train that middle layer parameter I can save the input of that layer and do forward from here, so that the training processs could be finished even faster. But I am not sure about how to implement this in pytorch. Do I have to rewrite the .forward function of the entire model， or there is a simpler way to achieve that?