I think your loss might grow pretty quickly, as you are summing the sample losses for a big batch of 7471 samples.
Try to use reduction='elementwise_mean' or maybe a smaller batch size and see, if you also get nan values.