Intro to Statistical Learning - Solutions for 2.1 I am reading An Introduction to Statistical Learning with Applications in R (ISLR) and I wonder what would be the answer for exercise 2.1 part (d). The question is, If the variance of the error terms $$\sigma^2 = \mathrm{Var}(\epsilon)$$ is extremely high, a more flexible method would do worse or better? My intuition is it does not matter what method we choose, as it is an irreducible error, but most solutions I found said that the high variance of error terms means that the sample will have a lot of noise in the relationship. Therefore we should prefer an inflexible method that is less likely to over-fit to this noise.
Can someone explain it to me?

It matters what you choose because a more flexible method may fit to the noise very easily, and you'll have to battle with it. As you mentioned, this is irreducible error, but an overfitted model will make much larger errors on the holdout set. Its aim is never reducing the irreducible error.
