Thanks ptrblck!


Makes sense!


Right, since not all optimizers have “nestorov” argument but it is evaluated in the beginning of SGD step( ), the setstate should ensure that the argument “nestorov” is set. However, what about the other parameters like “dampening”? It is also evaluated at the beginning of step( ), but not set in the setstate function. This would imply that all optimizers definitely have this argument…