I think your issue is specific to MLP regression via mini-batch gradient descent - I wouldn’t expect this bias from GLM/robust regression methods, esp. if training data is homoscedastic.
So, you have a hidden layer that is prone to “forget” outliers due to mini-batching, plus MLPs are biased to output values in region around zero (in fact, outputs like 10000 may even be unreachable with vanilla SGD and low learning rate). Taken together, these things may explain the underfitting in your plots.