PyTorch’s L-BFGS implementation doesn’t perform a line search, and I suspect that greatly hurts its performance. If you want, you can transfer everything to numpy and cally scipy’s fmin_lbfgs_b function. It looks like that Tensorflow implementation uses Scipy’s implementation under the hood anyways. Switching between PyTorch and Numpy might be slow though, and I don’t know of a Scipy wrapper like in tensorflow.