Is my regression network overfitting or is my simulated MRI training data unrepresentative of real MRI val/test data? I have been working on a regression task for the past one year and I am stuck. My project is to simulate voxel-wise MRI data using a physics-based function and train a neural network using that data to predict a continuous number (T2 of tissue). Essentially, my training set consists solely of simulated data and my validation/test sets consist solely of real MRI voxelwise data collected from healthy volunteers. Below is the learning curve I get when training a neural network with a linear activation unit as its output:

Below is the histogram of predicted T2 values from this model, when I tested it on the test set consisting of real MRI data. The model seems to mainly, but not only, predict one value.

I then tried training the same model architecture with only real data (i.e. training/val/test sets all consist of real MRI data from healthy volunteers). I got the following learning curve and histograms:


The histogram looks better now as it seems to predict a range. The model also seems to converge as the validation and training losses are low and have similar values. Based on this, is it the case that my model when trained on simulated data is overfitting and I need to introduce regularization (e.g dropout)? Or is it the case that, given that the model converges for the real data, my simulated data is simply not realistic enough when compared with the real data?
Would be grateful for any advice on how to progress further! Thanks!
Edit 1: I am going to try and implement a training-val set, which would contain some data from the original training set. The training-val and training sets would be independent data sets composed of simulated MRI data. Will try plotting that as well as the original validation set containing the real data to see if that can shed insight into whether this is a data mismatch problem. Will keep you posted!
Edit 2: Here is the histogram of the real and the simulated T2 ground truth data used to test and train the model respectively. Please note the following. Firstly, the simulated data has more values at higher ground truth T2s when compared to the real ground truth distribution because the real ground truth data only consists of healthy T2 values (which are lower) whereas the simulated ground truth data consists of both healthy and diseased T2 values (which are lower and higher, respectively). Secondly, the simulated, training data consists of 48,000 samples whereas the real validation and test data consists of 1,801 samples each (so 3,602 real data samples in total). The models will later be tested on diseased real data as well, but for now I only have access to data from healthy volunteers...


Firstly, you do seem to be getting some learning here, so your project looks good. The RMSE looks reasonable, since it is small compared to the squared values, but $R^2$ would be a better indicator.
I see that the model never predicts values above 45. Are they missing from the training data? If so, add them. If not, you seem to need to add more, or reconsider you loss function.
The tendency to predict one value often is odd, and is likely to due to either the training data or the loss function - I suggest checking them both.
