I have a model in which the input data has been scaled to be between [-1,1]. The corresponding targets are all values around -2700. I have initialized my model with various schemes, but currently utilizing the xavier initialization with a tanh activation function across 2 hidden layers (5 nodes each). It seems that when I try training my model with the default target values It is unable to train properly and steadies off at predictions being just the mean of all the targets. However, when I scale the targets to be between [0,1] the model trains properly. Can someone explain as to why I’m experiencing this? Is this normal or is something off? I was under the impression that target scaling was unnecessary. Thanks!