Hi,

First I can’t understand why is the gradient of the sum the same of the sum of the gradients for each sample in pytorch architecture.

This is happens because of the linearity of the derivatives:
d/dqqd(L0 + L1 + L2) = dL0/dqqd + dL1/dqqd + dL2/dqqd
But this assumes that L0, L1 and L2 are independent. So if in your forward, the result for each batch actually depends on the result from the others, you cannot separate the equation as L0 will depends on L1 etc.
Also errors of the order of 1e-5 are expected when doing these things and are simply due to the non-associativity of floating point numbers: because you compute things in a different order, you will get small errors. This is expected.