Making a shameless self plug, but if you’re still interested in this, I’ve just made public my implementation of L-BFGS which contains both a backtracking line search (with or without cubic interpolation) and weak Wolfe line search. Take a look here: https://github.com/hjmshi/PyTorch-LBFGS 258. Any feedback would be appreciated!