rawdev:

As far as I know, An output value of a node is need to backpropagate gradients through the node.
(e.g matmul gradient)
but in the upper case, the output value is not one.


I’m not sure what exactly this is referring to. In your code snippet you are reducing the losses by applying torch.sum on them and are calling total_loss.backward() later.
If you are concerned about the backward call on non-scalar values: by default PyTorch will pass a 1. gradient to backward(), if this operation is called on a scalar. If you are working with multiple values, you would need to specify the input gradient manually to the backward operation.