Hi everyone, I have the same problem, and my code is as following:
for batch_idx, (data, label) in enumerate(data_loader_new): 
    data, label = data.to(device), label.to(device)
    optimizer.zero_grad()
    output= model(data)
    Before = list(model.parameters())[0].clone()
    loss = loss_fn(pred=output, target=label)
    loss.backward(retain_graph=True)
    grad = torch.autograd.grad(outputs=loss, inputs=data) # I need it for some computations
    optimizer.step()
    After = list(model.parameters())[0].clone()
    print(torch.equal(Before.data, After.data))


It returns True.
I appreciate any help from your side.