<p>What about a block bootstrap approach? Block bootstrap is useful for time series because it preserves time series dependencies within the data, which traditional k-fold cross-validation does not preserve. In the following proposal, you would do the blocking but not the bootstrapping.</p>
<p>Let's say that out have 1,000 observations and use a time series approach that makes use of AR(5). You could break the data into 100 blocks of 10 observations. Next, you assign each block to one of five or ten groups. Then you do cross-validation using the left out 1/5th or 1/10th of the sample. Repeat for each of the five or 10 folds. You could also experiment with changing the size of the blocks and their starting points (1 to 20, 21-40, and etc versus 2-21, 22-41, and etc) to see if the results remain robust.</p>
<p>If you wanted to go for a block bootstrap instead of a k-fold approach, you could generate counter-factual data sets and estimate the model on the original data.  Then you could use the block bootstrapped data as the hold out sample on which to evaluate the model fit. Then, like in k-fold, you look for models with good in and out of sample performance.</p>

CC BY-SA 4.0
2024-10-27T14:56:49.743Z
2024-10-28T10:28:12.423Z
2024-10-28T10:28:12.423Z
BKay
<p>I have developed a model that predicts asset returns based on a training set. I'm aware that for i.i.d. data, best practices suggest using 5-fold or 10-fold cross-validation to balance the bias-variance tradeoff. However, with time series data, it's crucial to maintain temporal order, making cross-validation more complex.</p>
<p>I've encountered various suggestions in the literature and online. One common approach is to use an augmenting training window with a decreasing validation window. For instance, in the first fold, you could train on periods 1-48 and validate on 49-60, then for the second fold, train on 1-49 and validate on 50-60, and so on.</p>
<p>There are also methods that use a augmenting window, such as training on 1-49 and validating on 50, then training on 1-50 and validating on 51. Additionally, I've seen references to Monte Carlo cross-validation and other techniques.</p>
<p>I'm uncertain which cross-validation method would be best for my analysis. I've read that the first method I mentioned might be particularly effective for non-stationary data with structural breaks, which is likely in the case of asset returns. However, it seems there's no theoretical consensus on the optimal cross-validation approach for non-stationary time series data. Is that accurate, or is there a recommended method for my situation?</p>
<p>Thank you for your insights!</p>

CC BY-SA 4.0
2024-10-27T12:46:21.157Z
I think rolling or expanding training windows combined with a fixed forecast horizon (e.g. 1 period ahead) are the standard in time series forecasting literature. You describe an expanding window in your third paragraph.
Richard Hardy
<econometrics><time-series><stationary>
Best Practices for Time Series Cross-Validation: Which Method Should I Use?
dewewdew
# Best Practices for Time Series Cross-Validation: Which Method Should I Use?

(Asked by: username_0 on 2024-10-27)

I have developed a model that predicts asset returns based on a training set. I'm aware that for i.i.d. data, best practices suggest using 5-fold or 10-fold cross-validation to balance the bias-variance tradeoff. However, with time series data, it's crucial to maintain temporal order, making cross-validation more complex.

I've encountered various suggestions in the literature and online. One common approach is to use an augmenting training window with a decreasing validation window. For instance, in the first fold, you could train on periods 1-48 and validate on 49-60, then for the second fold, train on 1-49 and validate on 50-60, and so on.

There are also methods that use a augmenting window, such as training on 1-49 and validating on 50, then training on 1-50 and validating on 51. Additionally, I've seen references to Monte Carlo cross-validation and other techniques.

I'm uncertain which cross-validation method would be best for my analysis. I've read that the first method I mentioned might be particularly effective for non-stationary data with structural breaks, which is likely in the case of asset returns. However, it seems there's no theoretical consensus on the optimal cross-validation approach for non-stationary time series data. Is that accurate, or is there a recommended method for my situation?

Thank you for your insights!

**Tags:** econometrics, time-series, stationary

> **Comments:**
> **username_1**: I think rolling or expanding training windows combined with a fixed forecast horizon (e.g. 1 period ahead) are the standard in time series forecasting literature. You describe an expanding window in your third paragraph.  

---

## 1 answers

### Answer by username_2 on 2024-10-27. Score: 3

What about a block bootstrap approach? Block bootstrap is useful for time series because it preserves time series dependencies within the data, which traditional k-fold cross-validation does not preserve. In the following proposal, you would do the blocking but not the bootstrapping.

Let's say that out have 1,000 observations and use a time series approach that makes use of AR(5). You could break the data into 100 blocks of 10 observations. Next, you assign each block to one of five or ten groups. Then you do cross-validation using the left out 1/5th or 1/10th of the sample. Repeat for each of the five or 10 folds. You could also experiment with changing the size of the blocks and their starting points (1 to 20, 21-40, and etc versus 2-21, 22-41, and etc) to see if the results remain robust.

If you wanted to go for a block bootstrap instead of a k-fold approach, you could generate counter-factual data sets and estimate the model on the original data. Then you could use the block bootstrapped data as the hold out sample on which to evaluate the model fit. Then, like in k-fold, you look for models with good in and out of sample performance.


---

