Is it valid to cross-validate linear regression models out of chronological order in a time series context? For simplicity I'll stick to the univariate case with one exogenous regressor. Suppose I have
X = (x1, x2, ..., xn)
Y = (y1, y2, ..., yn)
where i < j indicates xi came before xj chronologically, and likewise for yi and yj, regardless of the unit of time, for 1 <= i < j <= n.
Leaving aside the usual caveats about the interpretability of a regression model fitted to time series data, is there anything philosophically incorrect about cross-validating my linear regression trained on these data out of order? For example, is it sound to do leave-one-out cross-validation with (xk, yk) as the test set, incorporating data (xt, yt) in the training set for t > k?
Given linear regression's distributional assumptions of linearity and constant variance, the order of the exogenous and endogenous coordinates doesn't matter within the same training set as long as pairwise indices are preserved (i.e. the ranks are kept the same, xk is only matched to yk and not some other yi).
It seems to follow that there should be no advantage to requiring that the test data strictly come after the training data for linear regression in particular, which has no notion of time or temporal order, if we are already making these distributional assumptions.

First, let's assume that your model is incorrect (as it always is).  Let's also assume that there is some systematic trend in the data, and that you omit the some observations in the middle of the data, choosing to train on data from the start and end of the process.
You risk having your model seem like it can predict the data better than it should in this case.  Because there is a systematic trend, the model will pick up that and your cross validation is really evaluating your model's ability to interpolate when you want to use the model to extrapolate.
Your validation strategy should reflect the way the model is to be used.  If you're using the model to extrapolate, then the validation strategy should be a sort of extrapolation.
