I’ve been trying to use this code. The _backtracking linesearch method doesn’t take bounds. Both _weak_wolfe and _goldstein take bounds, but don’t obey them. Further, the _max_alpha method breaks unless you are working with both upper and lower bounds. I’m not convinced that it calculates the max_alpha parameter correctly either.
I think that altering lbfgs.py 30 so that the step method can take a linesearch_function parameter (rather than throwing an error if it is not None) and then having the new __init__ and linesearch methods be the only new pieces of code in lbfgsb.py 59 would be a cleaner implementation.
Even more ideal, have the linesearch functions be in separate files within torch.optim and agnostic to lbfgs. Then have have the new __init__ be the only piece of code within lbfgsb.py 59 which allows the lbfgs step method to call the new linesearch methods with bounds.
I’m trying to work on a fix on my own, but I’m hoping that the author has already fixed these issues and could offer a new version to play around with.