Determining overfitting model by computing variance in prediction error I have a data set for regression, with a set of input features and 1 response variable. To confirm if a trained model has overfitted, we can see if the train error << test error at untrained sample points. However, I want to know if the following can also indicate an overfit model in case I have some idea of the extent of inherent noise in the response set. I simply compute the training errors (i.e. prediction error at each observation of my train set) and then observe the variance in the errors. If this variance is significantly less than the expected variance of the noise distribution (assuming noise to be white noise from a Gaussian distribution), then the model has overfit.
For example, I believe the hidden noise in my response data to be from the following distribution N(0,0.05). If the variance in my model prediction train errors is say 0.01, it clearly means that my model has learnt the noise in addition to the non-noisy signal.
Does this make sense!

It does make sense to me, but makes your decision rest on the accuracy of your assessment of the noise.
