Solved by AlphaBetaGamma96 in post #2 
              
              
                You could try copying the base class for SGD and just modify it to support negative momentum values? 
https://pytorch.org/docs/stable/_modules/torch/optim/sgd.html#SGD