To get the slope of the Loss function one has to get the derivative. I ve tested the following code with no success.
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time
for i in range(training_iter+1):
optimizer.zero_grad()
x_ = train_x.detach().requires_grad_(True)
output = model(train_x)
loss = -mll(output,train_y)
dy_dx = torch.autograd.grad(loss.mean(), x_,allow_unused=True)
loss.backward()
loss.backward(retain_graph=True) does not resolve the issue