Problem with this loop is that loss you are trying to backpropagate is still detached from graph.
I came up with idea once before to multiply dummy loss by zero, and add (calculated on the) side value, this way graph will stay intact, but now I think this operation will probably zero all gradients upstream.
Maybe there’s a way to replace value of calculated dummy loss inplace with new value, but not sure how, since torch doesn’t allow to modify values of leaf tensors inplace.