Answers.Body: <p>There is an implementation of the paper (&quot;Adversarial Sparse Transformer for Time Series Forecasting&quot;), in Python using Pytorch, <a href="https://github.com/hihihihiwsf/AST" rel="nofollow noreferrer">here</a>. Although it has the training and evaluation functionality implemented, it appears to be lacking a function for running a prediction.  Maybe you can fork it and extend it.</p>
  <p>UPDATE</p>
  <p>There is also a paper, <a href="https://arxiv.org/abs/2012.07436" rel="nofollow noreferrer">&quot;Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting&quot;</a>, by Zhou et al., which does forecasts on univariate and multivariate data.  Their code is <a href="https://github.com/zhouhaoyi/Informer2020" rel="nofollow noreferrer">here</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-24T16:15:57.047Z
Answers.LastActivityDate: 2021-03-25T18:49:24.430Z
Answers.LastEditDate: 2021-03-25T18:49:24.430Z
Answers.UserDisplayName: Brian O'Donnell
Body: <p>There is plenty of information describing Transformers in a lot of detail how to use them for NLP tasks. Transformers can be applied for time series forecasting.  See for example <a href="https://papers.nips.cc/paper/2020/file/c6b8c8d762da15fa8dbbdfb6baf9e260-Paper.pdf" rel="nofollow noreferrer">&quot;Adversarial Sparse Transformer for Time Series Forecasting&quot;</a> by Wu et al.</p>
  <p>For understanding it is best to replicate everything according to already existing examples. There is a very nice example for LSTM with flights dataset <a href="https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/" rel="nofollow noreferrer">https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/</a>.</p>
  <p>I guess I would like to know how to implement transformers for at first univariate (flight dataset) and later for multivariate time series data. What should be removed from the Transformer architecture to form a model that would predict time series?</p>
  
Tags: <deep-learning><transformer>
Title: How to construct Transformers to predict multidimensional time series?
UserDisplayName: Green
ThreadText: # How to construct Transformers to predict multidimensional time series?
  
  (Asked by: username_0 on 2021-03-24)
  
  There is plenty of information describing Transformers in a lot of detail how to use them for NLP tasks. Transformers can be applied for time series forecasting. See for example ["Adversarial Sparse Transformer for Time Series Forecasting"](https://papers.nips.cc/paper/2020/file/c6b8c8d762da15fa8dbbdfb6baf9e260-Paper.pdf) by Wu et al.
  
  For understanding it is best to replicate everything according to already existing examples. There is a very nice example for LSTM with flights dataset <https://stackabuse.com/time-series-prediction-using-lstm-with-pytorch-in-python/>.
  
  I guess I would like to know how to implement transformers for at first univariate (flight dataset) and later for multivariate time series data. What should be removed from the Transformer architecture to form a model that would predict time series?
  
  **Tags:** deep-learning, transformer
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-24. Score: 1 | Accepted answer
  
  There is an implementation of the paper ("Adversarial Sparse Transformer for Time Series Forecasting"), in Python using Pytorch, [here](https://github.com/hihihihiwsf/AST). Although it has the training and evaluation functionality implemented, it appears to be lacking a function for running a prediction. Maybe you can fork it and extend it.
  
  UPDATE
  
  There is also a paper, ["Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting"](https://arxiv.org/abs/2012.07436), by Zhou et al., which does forecasts on univariate and multivariate data. Their code is [here](https://github.com/zhouhaoyi/Informer2020).
  
  
  ---
  
  