BFGS & co are batch (whole dataset) optimizers, they do multiple steps on same inputs. Though docs illustrate them with an outer loop (mini-batches), that’s a bit unusual use, I think. Anyway, the inner loop enabled by ‘closure’ does parameter search with inputs fixed, it is not a stochastic gradient loop you do with SGD or Adam.