For illustration, here’s a toy model:
input = torch.distributions.normal.Normal(loc = 0, scale = 1).sample([50,5])
target = torch.ones([50, 1])
simple_model = torch.nn.Linear(5,1)
output = simple_model(input)

which I could optimise through standard procedures:
optimizer = optim.Adam(simple_model.parameters(), lr=1e-3)
criterion = torch.nn.MSELoss()
loss = criterion(output, target)
loss.backward()
optimizer.step()

However, my actual application is more complicated, and I deal with explicit gradients, not a loss. For example I only have access to the gradients returned by torch.autograd.grad(_, simple_model.parameters()). Is there anyway I can use these gradients to directly udpate the parameters of simple_model without writing my own optimiser?