Linear Regression : why expected value of response I am really confused... why linear regression is modelling the expected value of response(or conditional expected value)?
If we don't use mean square error as the loss function to minimise, is it still modeling E[Y]?
My understanding is that, we believe, there is a linear relation between X and Y. though in realization of Y (Y_hat), there are some error exist.
Y = f(X) + random error term
Y_hat = f(X)
Why/How does E[Y|X] has anything to do with this?
Thank you all in advance


If we don't use mean square error as the loss function to minimise, is it still modeling E[Y]?

Pretty much anything can be an estimator. For instance, we might choose to calculate the empirical median and use it as an estimator of the mean. To estimate variance, we might calculate $\frac{1}{n}\sum(x_i - \bar x)^2$, $\frac{1}{n-1}\sum(x_i - \bar x)^2$, or $\frac{1}{n+1}\sum(x_i - \bar x)^2$. Depending on the situation, each of these can be defended as an estimator of variance (MLE for a Gaussian, unbiased estimator, minimum MSE estimator for a Gaussian, respectively).
Consequently, you might choose to use the estimator calculated by minimizing absolute loss (explicitly models the conditional median, not the mean) or ridge loss, and you are perfectly within your rights to say that you are modeling (estimating) $\mathbb E[Y]$, (even if you are not doing so explicitly).
That might be a rather poor estimator of the conditional mean, but the OLS estimator might be a rather poor estimator of the mean, depending on the particulars of the problem.
