Specifically, by comparing them step by step, I saw that (gradient_1 + gradient_2 + gradient_3).norm() equals to gradient_4.norm(), however, (gradient_1 + gradient_2 + gradient_3).norm() == gradient_4.norm() is False. This is consistent after hundreds of iterations… Is it because of numerical instability? If so, does above code the right way to inspect different gradients by different losses?