For a  project that I have started to build in PyTorch, I would need to implement my own descent algorithm (a custom optimizer different from RMSProp, Adam, etc.). In TensorFlow, it seems to be possible to do so (https://towardsdatascience.com/custom-optimizer-in-tensorflow-d5b41f75644a 382) and I would like to know if it was also the case in PyTorch.
I have tried to do it by simply adding my descent vector to the leaf variable, but PyTorch didn’t agree:  “a leaf Variable that requires grad has been used in an in-place operation.”. When I don’t do the operation in-place, the “new” variable loses its position of leaf, so it doesn’t work neither…
Is there an easy way to create such a custom optimizer in PyTorch?
Thanks in advance