AR Forecasting one day ahead I am doing a project where I want to forecast prices for the next day based on some lagged values of the price. When I read about how to implement this (in Python) what I find it that they use a training data set, fit the model, and then use these parameters to forecast a test data set.
For example:
#first until n-5 observation
df_train = df['Temp'].iloc[:-5]
#n-5 until n observation
df_test = df['Temp'].iloc[-5:]

from statsmodels.tsa.ar_model import AutoReg
model = AutoReg(df_train, lags=22).fit()

forecasts = model.forecast(5).tolist()
test_values = df_test.tolist()
for index in range(len(forecasts)):
  difference = forecasts[index] - test_values[index]
  print(forecasts[index], test_values[index], difference)

Source
First, I see that long training sets are generally used, but then the model uses for example 22 lags to predict the test set, like here. In this case, wouldn't it be sufficient to use this training set:
df_train = df['Temp'].iloc[-27:-5]

Which would give 22 observations?
Secondly, in my case, I don't want to use a fixed training set to forecast a set number of days in the future, but I want to use my lagged data to predict the next 24 hours.
Say if I use a 7 days lag and the first forecasting day is n. Then, to predict n, I want to use observations n-8 until n-1. For the next day, n+1, I want to use n-7 until n to predict.
So:

*

*Do I even need a training and test split for this forecast?

*Are you aware of some sources where I can read about this procedure?

All the best,
Simon


First, I see that long training sets are generally used, but then the model uses for example 22 lags to predict the test set, like here. In this case, wouldn't it be sufficient to use this training set: df_train = df['Temp'].iloc[-27:-5] which would give 22 observations?

What you are suggesting is to use 22 data points* for estimating a model with 22 slope coefficients and an intercept. That is guaranteed to yield extreme overfitting. You would not even be able to use OLS as there are more free parameters than data points. You need a much larger training subsample.

I want to use my lagged data to predict the next 24 hours

If you have daily data, you cannot make hourly predictions. On the other hand, you probably meant forecasting 1 day ahead? That would work.

Say if I use a 7 days lag and the first forecasting day is n. Then, to predict n, I want to use observations n-8 until n-1. For the next day, n+1, I want to use n-7 until n to predict. So: Do I even need a training and test split for this forecast?

You need a training sample to estimate the model's coefficients. Having done that, you only need the most recent $p$ data points to predict the next one. (Here, $p$ is the order of the AR model you are using.)
*I am not proficient in Python, but does [-27:-5] not yield 23 data points?
