If the target range is so small, does it really matter that the loss is small too?
Since updates to the weights have to be small since the values are packed together in such a small range, I feel like the small loss might not be a problem (e.g. if the range was (0,10), the weights & loss might just be scaled up by 10 & 10^2 without changing learning performance much).
I tried to train with targets scaled up to (0,100) but it didn’t noticeably improve performance.