Solved by ptrblck in post #2 
              
              
                You could disable the gradient calculation via with torch.no_grad() as seen here: 
model1 = nn.Linear(1, 1)
model2 = nn.Linear(1, 1)

input = torch.randn(1, 1)
target = torch.randn(1, 1)
criterion = nn.MSELoss()
function = criterion

output1 = model1(input)

with torch.no_grad():
    output2 = model…