This time-series modeling winner used COCOB for RNN training. Can you show more detail on how to average the outputs of several optimizers in PyTorch?

  
      github.com
  
  
    Arturus/kaggle-web-traffic/blob/master/how_it_works.md 16
# How it works
__TL;DR__ this is seq2seq model with some additions to utilize year-to-year
and quarter-to-quarter seasonality in data.
___
There are two main information sources for prediction:
1. Local features. If we see a trend, we expect that it will continue
 (AutoRegressive model), if we see a traffic spike, it will gradually decay (Moving Average model),
 if wee see more traffic on holidays, we expect to have more traffic on
 holidays in the future (seasonal model).
2. Global features. If we look to autocorrelation plot, we'll notice strong
year-to-year autocorrelation and some quarter-to-quarter autocorrelation.

![autocorrelation](images/autocorr.png "Encoder-decoder")

The good model should use both global and local features, combining them
in a intelligent way.

I decided to use RNN seq2seq model for prediction, because:
1. RNN can be thought as a natural extension of well-studied ARIMA models, but much more
flexible and expressive.


  This file has been truncated. show original