How to cross-validate a time series LSTM model? I have been looking at how to split my data for training/validation/test for a timeseries using LSTM and came across: QA1 and QA2
Given I should implement walk-forward splits my depiction of it is:

Where each line is a Run followed by obtaining the best model.

*

*How should the best model be decided. More specifically, how is the test data used? do we evaluate at each "Run" with test data? i.e I use checkpoints to save the best epoch based on validation data.

*

*Do I use the best one?

*Do I need to evaluate all the models/epochs against test data and take the best of these?

*or?



*On the final "Run", what is the construct? Do we have no validation or no test data?


*

*If we don't have validation data, is it possible to run model.fit without validation data?



*

*If we don't have test data, do we not have a final evaluation?




This is nested cross validation (CV). The test data is used to estimate the error of that run. Then, you average the errors obtained over each run's test data. This completes the outer part of CV. Its purpose is to estimate the real world performance of your procedure.
The validaton data is for tuning the hyper-parameters. In your case, this can be number of epochs, number of hidden dimensions etc. The best performing model on the validation fold is selected to estimate the performance on the test set.
In the final run, where you're going to deploy the model, it's still preferable to have train+validation+test split, so that you can still optimize the hyper-parameters and do a final check on the test data.
