Machine Learning, Simulation, Physics.
most of these methods are not available as built-in optimizers in PyTorch, nor in most deep learning frameworks. At some point, I implemented a Hessian-free conjugate gradient optimizer in PyTorch, but I didn’t polish or test it enough, so we’ll have to live with gradient descent with backtracking