@Oli, ah I see, anyway, I meet the same problem but I figure out the problem is because I used BatchNorm1d before Linear layer, maybe it scale the output too much so that the final Linear scale it up to nearly similar value.
Anyway, have you ever try the CNNs model for a Regression problem, I try to train some models but it seem like they tend to return the variable in a fixed range (so it will miss some too big or too small targets)