Hi @Ginsunuva,
Thank you for your reply, I would like to learn more about it.
For example, assuming the pre-trained model is a weighted MSE loss,
,
where  x is the input, y is the target, alpha is a parameter of the pre-trained model and f is the model I want to train, according to your answer, we still have to compute dL/df, but we don’t have to compute dL/d(alpha), so we can save GPU memory via this way.
Do I understand correctly ?