Making a shameless self plug, but I just released my implementation of L-BFGS (based on various codes, including Mark Schmidt’s minFunc, the original PyTorch implementation, and Michael Overton’s weak Wolfe line search). It addresses some of the issues you all mention here and is also compatible with recent advancements in stochastic quasi-Newton methods, such as multi-batch and Schraudolph’s approaches for performing quasi-Newton updating, as well as Powell damping. Feel free to take a look, and any feedback would be appreciated!
Link is here: https://github.com/hjmshi/PyTorch-LBFGS 145.