Answers.Body: <p>Transformers, being a general-purpose sequence model can be used for Time-Series forecasting.</p>
  <p>There are some <a href="https://arxiv.org/abs/2001.08317" rel="nofollow noreferrer">papers</a> dedicated to the use of Transformer for time-series prediction and <a href="https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630" rel="nofollow noreferrer">blogs</a>.</p>
  <p>The main ingredient for the autoregression in predictions is the mask in Transformer encoder.
  When the next element is predicted, tokens in the sequence attend only to the tokens back in time.</p>
  <p><a href="https://i.sstatic.net/cdn1t.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/cdn1t.png" alt="enter image description here" /></a></p>
  <p>After each block a new element is predicted, based on the decoder and encoder tokens.</p>
  <p>However, since the dimensionality of your data seems to be rather small, I would suggest starting from something simpler - say linear AR models or RNN, and only then work with transformers.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-19T09:53:31.037Z
Answers.LastActivityDate: 2021-10-19T09:53:31.037Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>Can I use the transformers for the prediction of wind power with the historical data?</p>
  <p>Dataset</p>
  <p>Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind Direction (Degree), Wind Speed at 76.8 m (m/sec), Power Generated(kW).</p>
  <p>15 years of data from 2007 to 2021 with a sampling time of 1 hour​</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-19T12:23:27.830Z
Comments.Text: Hello. Welcome to AI SE. You provide some info about your dataset, but it's not clear what are your features (or independent variables) and what the label (or dependent variable, i.e what you want to predict) is. You should probably also tell us how large your dataset is in terms of observations. Finally, you should explain why you think the transformer may be a good choice for this.
Comments.UserDisplayName: nbro
Tags: <neural-networks><deep-learning><datasets><transformer>
Title: Can I use the transformers for the prediction of historical data?
UserDisplayName: Dinash .R.R
ThreadText: # Can I use the transformers for the prediction of historical data?
  
  (Asked by: username_0 on 2021-10-19)
  
  Can I use the transformers for the prediction of wind power with the historical data?
  
  Dataset
  
  Datetime, Ambient temperature (Degree), Dewpoint (Degree), Relative Humidity\n (%), Air Pressure, Wind Direction (Degree), Wind Speed at 76.8 m (m/sec), Power Generated(kW).
  
  15 years of data from 2007 to 2021 with a sampling time of 1 hour​
  
  **Tags:** neural-networks, deep-learning, datasets, transformer
  
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE. You provide some info about your dataset, but it's not clear what are your features (or independent variables) and what the label (or dependent variable, i.e what you want to predict) is. You should probably also tell us how large your dataset is in terms of observations. Finally, you should explain why you think the transformer may be a good choice for this.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-10-19. Score: 1
  
  Transformers, being a general-purpose sequence model can be used for Time-Series forecasting.
  
  There are some [papers](https://arxiv.org/abs/2001.08317) dedicated to the use of Transformer for time-series prediction and [blogs](https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630).
  
  The main ingredient for the autoregression in predictions is the mask in Transformer encoder.
  When the next element is predicted, tokens in the sequence attend only to the tokens back in time.
  
  [![enter image description here](https://i.sstatic.net/cdn1t.png)](https://i.sstatic.net/cdn1t.png)
  
  After each block a new element is predicted, based on the decoder and encoder tokens.
  
  However, since the dimensionality of your data seems to be rather small, I would suggest starting from something simpler - say linear AR models or RNN, and only then work with transformers.
  
  
  ---
  
  