Yes. I tried to print the sum of the weights of the fc2 layer after each .step and every time I got the different sum.
# update params
        optimizer.step()

print(torch.sum(main_model.fc1.weight.data).item())

gives
14.266899108886719
16.328222274780273
16.12299346923828
15.318760871887207
14.778365135192871
14.173827171325684
13.708841323852539
13.355201721191406
12.91303825378418
12.379934310913086
11.765584945678711
11.129434585571289
…