Intro to Statistical Learning - Solutions for 2.1 I am reading An Introduction to Statistical Learning with Applications in R (ISLR) and I wonder what would be the answer for exercise 2.1 part (d). The question is, If the variance of the error terms $$\sigma^2 = \mathrm{Var}(\epsilon)$$ is extremely high, a more flexible method would do worse or better? My intuition is it does not matter what method we choose, as it is an irreducible error, but most solutions I found said that the high variance of error terms means that the sample will have a lot of noise in the relationship. Therefore we should prefer an inflexible method that is less likely to over-fit to this noise.
Can someone explain it to me?

I would propose a thought experiment. Lets say you have some data generating process y=x + e with e a stochastic disturbance term with some distribution with a really high variance and x some number between 0 and 10. A very inflexible model would be y=constant=E[y], this would not even take x into account. You just 'guess' the mean of y every time.
Now imagine the opposite, you estimate an extremely flexible model y=f(X,B)+e. This model will fit every x to every y in your sample exactly. A perfect fit. Now we know the data generating process. We know every point y is mostly determined by your disturbance term e, since its value is probably much more extreme than the value of x. But your flexible model will attribute all variation in y to variation in X. Whereas in reality y is just very noisy without much structure. Attributing all variation to x is overfitting.
You ask "... a more flexible method would do worse or better?", I take it that by "better" you mean predicting y from new observations of x, given that y is generated by the same process. The inflexible method would predict E[y] whereas the flexible method would be all over the place.
