I use jupyter notebook to inspect the result:
#Test code
L2Loss = nn.MSELoss()
outputs,losses = model(batch_data, batch_cat)
optimizer = torch.optim.Adam(model.parameters(), lr=1)
los = L2Loss(layers[0],torch.zeros_like(layers[0]))
print(los)
los.backward()
optimizer.step()
print(los)

First time I run the code it output the same value of ‘los’, and the parameters of model don’t change. And when I run the code again, the error appears that “Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.” It’s probably because the grad didn’t back propagate correctly? I just want it updated with respect to L2 loss.