Considering the docs 59 I realized that every optimizer.step method has an (optional) closure argument (for LBFGS it’s even required). The info text is:

A closure that reevaluates the model and returns the loss.

For me it is not completely clear which steps should be taken in this closure function. I reviewed some examples (for example here 120 and there 68) and it looked like that within the closure we should:

Zero the gradients
Compute the loss
Backprop on the loss
Return the loss

So my questions are:

Is the above list of (minimum required) steps that should be taken within the closure function correct and complete?
What is the purpose of returning the loss? What does the optimizer do with it? Should we return the loss that we backproped on or create a new one (the two examples handle this situation differently; maybe it doesn’t matter)?
For an optimizer for which the closure argument is optional, is closure(); optimizer.step() similar to optimizer.step(closure)?

Thanks for your input!