Solved by albanD in post #2 
              
              
                Hi, 

First I can’t understand why is the gradient of the sum the same of the sum of the gradients for each sample in pytorch architecture. 

This is happens because of the linearity of the derivatives: 
d/dqqd(L0 + L1 + L2) = dL0/dqqd + dL1/dqqd + dL2/dqqd 
But this assumes that L0, L1 and L2 are i…