In the CAFFE, it has

Screenshot from 2018-06-12 10-39-46.png1167×578 62.6 KB

We can find the setting in https://github.com/BVLC/caffe/blob/master/examples/mnist/lenet_solver_adam.prototxt#L12 28
While in pytorch, the Adam method is
class torch.optim.Adam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)

I did not find any params for momentum. How to set it in pytorch? Thanks