Model and CV setup for structural timeseries forecast
I wrote this question on cross-validated, but no replies were yet made, so I'm posting here as well since the topic may overlap a little bit. I have no problem with deleting one off one of the sites, if it is in violation of community guidelines.
I'm attempting to use tensorflow-probability's structural timeseries forecast tools, but I'm running into memory allocation problems and the trainings seem to last quite long, so I'd like to check if I'm maybe using the framework in a way it's not meant to be used.
So here's the setup of the problem - I have hourly electricity consumption and meteorological data from 2013 to 2019. I would like to create a model that can perform day-ahead consumption forecasts. 650
The model components I use are various seasonal effects (hour, weekday, month), autoregressive and a linear regression for the meteorological data (air temperatures). The solution I've set up so far looks as in this issue comment, so the build function is something like:
```python def build_model(observed_time_series, x_effect_design_matrix):
hour_of_day_effect = sts.Seasonal(...)
day_of_week_effect = sts.Seasonal(...)
month_of_year_effect = tfp.sts.Seasonal(...)
autoregressive = sts.Autoregressive(...)
x_effect = sts.LinearRegression(
design_matrix = x_effect_design_matrix,
weights_prior = tfd.Normal(loc=0., scale=1.),
name='x_effect')
model = sts.Sum([hour_of_day_effect,
day_of_week_effect,
month_of_year_effect,
x_effect,
autoregressive],
observed_time_series=observed_time_series)
return model
```
and it's used as:
```python
get data
training_time_series, training_x_effect = get_training_data() forecast_time_series, forecast_x_effect = get_forecast_data()
train model
training_model = build_model(training_time_series, training_x_effect) q_samples_ = train_model(training_model)
forecast
forecast_model = build_model(forecast_time_series, forecast_x_effect) forecast_dist = tfp.sts.forecast( model=forecast_model, observed_time_series=forecast_time_series, parameter_samples=q_samples_, num_steps_forecast=num_forecast_steps) ```
(The training function just uses variational inference, but that part executes more-or-less well, a little slow, but it goes through it)
What bugs me is, after I've trained the model and gotten the parameter samples, what needs to be passed in the forecast_time_series and forecast_x_effect variables? For instance, I've fit the model to hourly data from year 2013 and want to make a forecast for date 2014-05-15. What data needs to be contained in the variables in question? Since the model has components that make use, for instance, the month when data was observed, does that mean that the data passed needs to be from the beginning of the year? (how else is the model supposed to "know" which month it is) But then I get out-of-memory errors and it makes me think that maybe this isn't the way? 613