Answers.Body: <p>In simple terms, without this transformation the network doesn't know that 2022-01-01 data likely correlates with the January 1st on previous (and future!) years. And actually it is likely to also correlate with January 3rd, and maybe even December 29th.</p>
  <p>It is also possible to encode the day of year as float between 0.0 and 1.0 (first day of year and last day of year), and that would also provide this &quot;correlation&quot; information. However it isn't as good, since 2021-12-31 would have a value of 1.0, very far from the next day's 2022-01-01 value of 0.0.</p>
  <p>Yet an other option is to treat the day of year as a categorical value between 1 and 365 (except on leap years, which is annoying). This would provide some degree of information, but it would lose the apriori knowledge that the 10th day of the year correlates more with the 11th than 100th.</p>
  <p>Splitting this to sine and cosine signals circumvents these issues. (now I'm thinking whether this got too verbose.)</p>
  <p>This same logic applies to the time of day, 23:59:59 should be very correlated with 00:00:00.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-06T14:30:00.543Z
Answers.LastActivityDate: 2022-01-06T14:30:00.543Z
Answers.UserDisplayName: NikoNyrh
Answers.Body: <p>Sorry for my weak English. Your are using neural network to forecast times series which often have irregular fluctuations.</p>
  <p>Stock values are volatile and have changing frequency. Applying periodic encodings to original data makes it easier to capture frequency information.</p>
  <p>Read this paper to understand why it is essential for the neural network to know about frequency of data and how sinusoidal encoding could make it easier for the neural network to predict highly varying patterns inside data.</p>
  <p><a href="https://bmild.github.io/fourfeat/" rel="nofollow noreferrer">https://bmild.github.io/fourfeat/</a></p>
  <p>Or you could use periodic activation functions without applying sinusoidal encoding. Networks with non-periodic activation fail to predict time series.</p>
  <p><a href="https://arxiv.org/pdf/2006.08195.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2006.08195.pdf</a></p>
  <p>Deep networks are biased towards low frequency functions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-31T22:31:32.860Z
Answers.Text: Thank you for the papers, I'll check them out!
Answers.UserDisplayName: python newbie
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-31T21:53:00.167Z
Answers.LastActivityDate: 2021-12-31T22:17:57.603Z
Answers.LastEditDate: 2021-12-31T22:17:57.603Z
Answers.UserDisplayName: bitWise
Body: <p>I'm trying to learn how time series forecasting models work and while reading a tutorial off the TensorFlow website I came across these algorithms. I don't quite understand what the article means by &quot;time signals&quot; and how do sine and cosine functions help accomplish them. Can anyone please explain?</p>
  <p><a href="https://tensorflow.org/tutorials/structured_data/time_series" rel="nofollow noreferrer">Here's a link to the tutorial</a></p>
  <p>The following code was provided along with the caption</p>
  <p>&quot;the time in seconds is not a useful model input. Being weather data, it has clear daily and yearly periodicity. There are many ways you could deal with periodicity.</p>
  <p>You can get usable signals by using sine and cosine transforms to clear &quot;Time of day&quot; and &quot;Time of year&quot; signals:&quot;</p>
  <pre><code>day = 24*60*60
  year = (365.2425)*day
  
  df['Day sin'] = np.sin(timestamp_s * (2 * np.pi / day))
  df['Day cos'] = np.cos(timestamp_s * (2 * np.pi / day))
  df['Year sin'] = np.sin(timestamp_s * (2 * np.pi / year))
  df['Year cos'] = np.cos(timestamp_s * (2 * np.pi / year))
  </code></pre>
  
Tags: <tensorflow><time-series>
Title: How do sine and cosine transforms help in extracting frequencies in time series forecasting models?
UserDisplayName: python newbie
ThreadText: # How do sine and cosine transforms help in extracting frequencies in time series forecasting models?
  
  (Asked by: username_0 on 2021-12-31)
  
  I'm trying to learn how time series forecasting models work and while reading a tutorial off the TensorFlow website I came across these algorithms. I don't quite understand what the article means by "time signals" and how do sine and cosine functions help accomplish them. Can anyone please explain?
  Here's a link to the tutorial
  The following code was provided along with the caption
  "the time in seconds is not a useful model input. Being weather data, it has clear daily and yearly periodicity. There are many ways you could deal with periodicity.
  You can get usable signals by using sine and cosine transforms to clear "Time of day" and "Time of year" signals:"
  day = 24*60*60
  year = (365.2425)*day
  
  df['Day sin'] = np.sin(timestamp_s * (2 * np.pi / day))
  df['Day cos'] = np.cos(timestamp_s * (2 * np.pi / day))
  df['Year sin'] = np.sin(timestamp_s * (2 * np.pi / year))
  df['Year cos'] = np.cos(timestamp_s * (2 * np.pi / year))
  
  **Tags:** tensorflow, time-series
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-01-06. Score: 1
  
  In simple terms, without this transformation the network doesn't know that 2022-01-01 data likely correlates with the January 1st on previous (and future!) years. And actually it is likely to also correlate with January 3rd, and maybe even December 29th.
  
  It is also possible to encode the day of year as float between 0.0 and 1.0 (first day of year and last day of year), and that would also provide this "correlation" information. However it isn't as good, since 2021-12-31 would have a value of 1.0, very far from the next day's 2022-01-01 value of 0.0.
  
  Yet an other option is to treat the day of year as a categorical value between 1 and 365 (except on leap years, which is annoying). This would provide some degree of information, but it would lose the apriori knowledge that the 10th day of the year correlates more with the 11th than 100th.
  
  Splitting this to sine and cosine signals circumvents these issues. (now I'm thinking whether this got too verbose.)
  
  This same logic applies to the time of day, 23:59:59 should be very correlated with 00:00:00.
  
  
  ---
  
  ### Answer by username_2 on 2021-12-31. Score: 0
  
  Sorry for my weak English. Your are using neural network to forecast times series which often have irregular fluctuations.
  
  Stock values are volatile and have changing frequency. Applying periodic encodings to original data makes it easier to capture frequency information.
  
  Read this paper to understand why it is essential for the neural network to know about frequency of data and how sinusoidal encoding could make it easier for the neural network to predict highly varying patterns inside data.
  
  <https://bmild.github.io/fourfeat/>
  
  Or you could use periodic activation functions without applying sinusoidal encoding. Networks with non-periodic activation fail to predict time series.
  
  <https://arxiv.org/pdf/2006.08195.pdf>
  
  Deep networks are biased towards low frequency functions.
  
  > **Comments:**
  > **username_0**: Thank you for the papers, I'll check them out!  
  
  ---
  
  