Hi,
The thing is that by this argument, all optimizer names are confusing.
The optimizers provided in pytorch actually implement the update steps corresponding to the algorithm that is used for their name.