I’m not familiar with your use case, but the optimizer would already update all passed parameters in the optimizer.step() method. You are currently subtracting the alpha*grad again from each parameter even after the parameters were already updated.
Could you explain your use case a bit more, as I might misunderstand it?