I’m not completely sure about the internals but I’ve used closure that way successfully. So to comment on my own questions:

I performed those steps in the closure and it worked.
Some optimizers (e.g. LBFGS 80) will terminate depending on the value of the loss, so that’s why it needs access to it and hence it should be returned.
I suppose those two versions are similar.

Some optimizers (again LBFGS) need to evaluate the model multiple times hence the closure definition makes that possible.
Anyway I would be glad if someone with more in-depth knowledge could comment on these questions and clarify the situation.