ttsesm:

Weighting though I am not sure yet why it didn’t help, since in principle it should improve.


Well, that’s not proper oversampling - if you would show the network an “outlier” 100 times, 100 small gradient steps would have changing directions, instead of big linear jumps.



 ttsesm:

Another idea that came to my mind is whether it would be better (or it is possible) if I create a separate loss output for each cluster and then either combine them to an overall loss or somehow have a different regression branch for each cluster in the architecture.


If you rule out numerous “configuration” issues, this leaves “forgetfulness” and “limited memory” as causes for big final errors (for training set, for validation set you may also experience poor generalization). In that case, architecture tweaks with specific intents may indeed help.
I’d suggest to try attention as a way to have non-shared parameter values and deal with “forgetfulness”.
Another (a bit similar) thing to try would be to train network to output quantiles of some  distribution approximating your whole dataset (log-normal perhaps), with non-trainable (0,1)->Y_domain final transform.