The scenario is the latter of the cases. I have already tried scaling the targets, but as you mentioned - the small large values dominate and the smaller values despite being the majority are treated as noise. Not too sure if batch normalization would help here? Or possibly working in logspace?