The .backward() computes the gradients, it’s the optimizer .step() that do update of the weights that were given to it when it is created.
To get the gradients you can do:
inp = # Some input
label = # Some label
inp.requires_grad_()

out = model(inp)
loss = criterion(out, label)

# Do a regular gradient step from the optimizer
optimizer.zero_grad()
loss.backward()
optimizer.step()

# Grab input gradient
inp_grad = inp.grad