Standardization and test prediction in linear regression I am considering a linear regression model to predict a target $y\in\mathbb{R}^n$ from a data matrix $X\in\mathbb{R}^{n\times d}$. Let $X_1,\ldots,X_d\in\mathbb{R}^n$ be the columns of $X$. The prediction of $y$ is $\hat{\alpha}1_n+X\hat{\beta}$, where $$\hat{\alpha},\hat{\beta}\in\arg\min_{\alpha\in\mathbb{R}\\\beta\in\mathbb{R}^d}\|y-\alpha 1_n-X\beta\|_2^2.$$

*

*It is common practice to standardize the columns of $X$ to obtain a new matrix $X'$, for, e.g., interpretability: the columns of $X'$ are, for all $j\in\{1,\ldots,d\}$, $$X_j'=\frac{1}{\sigma_j}(X_j-\mu_j1_n),\quad\text{where}\quad\mu_j=\frac{1}{n}\sum_{i=1}^nX_{i,j}\quad\text{and}\quad\sigma_j=\frac{1}{n}\sum_{i=1}^nX_{i,j}^2-\mu_j^2.$$ Then the prediction of $y$ is $\hat{\alpha}'1_n+X'\hat{\beta}'$, where $$\hat{\alpha}',\hat{\beta}'\in\arg\min_{\alpha\in\mathbb{R}\\\beta\in\mathbb{R}^d}\|y-\alpha 1_n-X'\beta\|_2^2.$$ While I understand that this does not affect the prediction accuracy of $y$, what about the prediction of a test response? Given a test data matrix $X_\text{test}\in\mathbb{R}^{m\times d}$, the predicted response is $\hat{\alpha}'1_m+X_\text{test}'\hat{\beta}'$, where $X_{\text{test},j}'=(1/\sigma_j)(X_{\text{test},j}-\mu_j1_m)$. Without standardization, the predicted response is $\hat{\alpha}1_m+X_\text{test}\hat{\beta}$, which seems more direct and probably more accurate. Are there situations where we are still better off using standardization for test prediction?


*Suppose I also standardize $y$ to obtain a new target $y'$, which is not necessarily useful (or is it?) but not incorrect either. Then the prediction of $y'$ is $\hat{\alpha}''1_n+X'\hat{\beta}''$, where $$\hat{\alpha}'',\hat{\beta}''\in\arg\min_{\alpha\in\mathbb{R}\\\beta\in\mathbb{R}^d}\|y'-\alpha 1_n-X'\beta\|_2^2,$$ and the prediction of $y$ is $\sigma_y(\hat{\alpha}''1_n+X'\hat{\beta}'')+\mu_y1_n$. By the way, we know that $\hat{\alpha}''=0$ (Why does the y-intercept of a linear model disappear when I standardize variables?). Is the test prediction $\sigma_y(\hat{\alpha}''1_m+X_\text{test}'\hat{\beta}'')+\mu_y1_m$? How does its accuracy compare to that in 1.?
Thanks!

The big rule for these transformations is that you learn them from the test set and then apply those to the out-of-sample data. If your standardization of in-sample $x$ involves subtracting a number from every value and then dividing by another number, use those same number when you standardize out-of-sample $x$. Do not calculate those numbers from the out-of-sample data.
Ditto for $y$.
If you do something to the training data, you must do that same transformation to the testing data! If you get coefficient estimates for standardized data and then test with unstandardized data, those coefficient estimates apply for standardized inputs, not unstandardized inputs.
