You don’t have to set requires_grad = False if you don’t have an optimizer containing these parameters. Depending on your case, you could eventually wrap the calls of Model1 with with torch.no_grad() since your version still tracks the operations performed by the first model