TanmDL:

I have a fundamental question regarding the loss backward. The final loss is the summations of all losses. I think scalar loss value will update the parameters of individual models that are connected by the closed graph of specific input labels. Is the assumption correct?


I think your explanation is correct, although I’m not sure, what “closed” graph means exactly.
However, each operation in the forward pass will create a computation graph, which will be used to calculate the gradients for the involved parameters in the backward call.