Simple Linear Regression Question: How does correlation between X and Y affect MSE? Suppose we know that the correlation between $X_1$ and $Y$ is $\rho_{X_1Y}$ and the correlation between $X_2$ and $Y$ is $\rho_{X_2Y}$. Furthermore, suppose $0 < \rho_{X_2Y} < \rho_{X_1Y}$. Now, we fit the simple linear regression models $Y \sim X_1$ and $Y \sim X_2$, which one will have a lower MSE? i.e. which one has a lower $\parallel Y - \hat{Y} \parallel_2^2 \hspace{0.2cm} = \hspace{0.2cm} \parallel \hat{\epsilon} \parallel_2^2$
I am quite clueless. Is it even possible to say anything about MSE from just the correlation?

In simple linear regression, the squared correlation between the $X$ variable and $Y$ variable is equal to the $R^2$.
Also, $R^2$ has an equivalent representation involving MSE:
$$
R^2=1-\dfrac{MSE}{Var(Y)}
$$
In both of your simple linear regression equations, the $Y$ is the same, so the $Var(Y)$ in the denominator is the same for each. Consequently, the greater correlation magnitude corresponds to the higher $R^2$ and thus the lower $MSE$, with the extreme being an $MSE$ of zero (correlation magnitude equal to $1$, and $R^2=1$).
This makes sense to me. A high correlation (magnitude) between the predictor and outcome should correspond to a small amount of error.
