Solved by ptrblck in post #2 
              
              
                The result should be the same, if you use the sum as the reduction type (reduction='sum' for nn.MSELoss). 
Thus it depends also what reduction you are using in your custom loss function. 
Here is an example code snippet using nn.CrossEntropyLoss. 
Note that I called model.eval() to get the same outp…