Ignite’s Metric API allows to inspect what happens batch by batch. All metrics allow to do the following:
mse_metric = MeanSquaredError()

mse_metric.reset()

mse_metric.update((y_pred1, y1))
# check result of the 1st batch
print(mse_metric.compute())

mse_metric.update((y_pred2, y2))
# check result of the 1st and 2nd batch
print(mse_metric.compute())

In this way you can compare both on your predictions and targets and see where comes from the difference.
HTH
PS.
Shouldn’t loss_fn = torch.nn.MSELoss() have reduction “sum” ?