Hi together,
the documentation to the SGD with momentum method emphasizes that PyTorch uses a different iteration scheme compared to the original one introduced in the scientific literature. Is there a reason for this?
I care since I am playing with an algorithm that builds on the original momentum method and I would like to use the latter instead of PyTorch’s version. I can probably just edit the optimizer source code myself, but I was wondering about the reason behind the change.
https://pytorch.org/docs/stable/generated/torch.optim.SGD.html#torch.optim.SGD 5
Best,
PiF