If I’m not mistaken, the loss for a classification task using nn.MSELoss will be scaled down the closer the output gets to the target, which might slow down the training towards the end.
@rasbt derives the formula here and you can find some additional information in this blog post (at the end).