Might be unrelated, but could you check if the model output and target have the same shape?
If your target is missing the feature dimension ([batch_size] instead of [batch_size, 1]), an unwanted broadcast might be applied.
In the latest version this should throw a warning.