See LBFGS.step.
There are  lines as follows:
orig_loss = closure()
loss = float(orig_loss)

It expects that output of closure to be a number.
Other methods are different in implementation of step function.