Is there any optimizer in pytorch that currently supports line search? I want to solve an optimization problem where being able to guarantee that the error always goes down is necessary. Last time I checked, LBFGS for example, in theory supports line search, but it’s not implemented yet in pytorch. Is there any optimizer that supports line search?