Answers.Body: <p>A standard method for pre-processing time series data for neural network architectures, such as an LSTM, is to normalize the data. Good tutorials will include this step. There are several variations to normalizing time series data. Jason Brownlee writes a lot of good stuff on machine learning. Please see his article on <a href="https://machinelearningmastery.com/how-to-scale-data-for-long-short-term-memory-networks-in-python/" rel="nofollow noreferrer">How to Scale Data for Long Short-Term Memory Networks in Python</a>.</p>
  <p>Also, see the replies/answers to: <a href="https://stackoverflow.com/questions/43467597/should-i-normalize-my-features-before-throwing-them-into-rnn">https://stackoverflow.com/questions/43467597/should-i-normalize-my-features-before-throwing-them-into-rnn</a></p>
  <p>The following is a tutorial on predicting stock prices, employing normalization, using an LSTM: <a href="https://towardsdatascience.com/time-series-forecasting-predicting-stock-prices-using-an-lstm-model-d0056cd5e055" rel="nofollow noreferrer">Time-Series Forecasting: Predicting Stock Prices Using An LSTM Model</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-12T05:27:35.990Z
Answers.Text: Thank you for your reply. It was a great helps to understand the LSTM using more reasonable data and model structure. I will carefully read the article that you recommended.
Answers.UserDisplayName: orde.r
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-09T18:32:43.377Z
Answers.LastActivityDate: 2023-06-09T18:32:43.377Z
Answers.UserDisplayName: Brian O'Donnell
Body: <p>I just started to study time-series forecasting using RNN.</p>
  <p>I have a few months of time series data that was an hour unit.
  The data is a kind of percentage value of my little experiment and I would like to forecast the future condition of this.
  Many tutorials and web info introduced direct training and forecasting the time series data without any data pre-processing.</p>
  <p>But for the RNN (or ML and DL), I think we should consider the data's condition that is stationary or not.</p>
  <p>My data is totally random condition which is stationary data (no seasonality, no trend).</p>
  <p>For example, the US stock prediction tutorial showed super great accuracy forecasting performance according to many LSTM tutorials.
  <em>[If this really works and is true, then all ML developers will be rich.]</em></p>
  <p>And, Some of them didn't emphasize and note a kind of the data pre-processing such as non-stationary to stationary something like that.</p>
  <p>According to my short knowledge, I think the non-stationary data such as stock price (will have trend) should be converted as a stationary format through differencing or some other steps. and I think this is a correct prediction as a view of theoretical sense even if the accuracy is not high.</p>
  <p>So my point is, I'm a bit confused about whether that really is no need for any preprocessing to treat stationary or not.</p>
  <p>For my case, I applied differencing step (<span class="math-container">$t_n - t_{n-1}$</span>)to my time-series data in order to remove the trend or some periodic situation.</p>
  <p>Is my understanding not correct?</p>
  <p>Why do time-series forecasting tutorials not introduced data stationarity?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-09T18:10:31.657Z
Comments.Text: You stated that your data is 'totally random'. How do you know it is 'totally random'? Truly random time series data can not be forecasted. Stock price data is not random although it has some degree of randomness.
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-09T18:11:55.777Z
Comments.Text: You mentioned stock data and 'future condition'. That is very unclear. Are you trying to predict the future price of a stock given historical stock price data?
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-09T18:21:23.633Z
Comments.Text: You mentioned that a tutorial shows high forecasting accuracy. Please cite the tutorial. You pointed out that ML developers would be wealthy if high accuracy was possible. The fact is, practically all tutorials and papers do not report a valid time series forecasting metric! RMSE and other metrics that lose the directional accuracy of the forecast are useless for forecasting direction error. Another important fact is that LSTMs alone do not provide an accuracy suitable for forecasting stock prices for profit for long time regimes.
Comments.UserDisplayName: Brian O'Donnell
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-12T05:15:11.707Z
Comments.Text: Sorry for the unclear post. Actually, My data is a CPU Usage history from my lab top. This was a kind of practice work. According to my data, I found It seems no periodic situation or trend. So that is the reason why I said 'My data is totally random' but I think I misunderstood the random data such as stationary. the stock is one example to explain my question.
Comments.UserDisplayName: orde.r
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-12T05:24:40.730Z
Comments.Text: Yes. Basically, I would like to predict future CPU Usage using CPU usage history as same as stock price prediction. In this step, not only 1 unit prediction a period prediction I want to do.
Comments.UserDisplayName: orde.r
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-12T05:26:06.443Z
Comments.Text: I saw a few tutorial blogs and webpage that introduced 'LSTM for the time-series data'. most of the page is not written English (I'm sorry) and here is one of a page that I saw before. https://towardsdatascience.com/aifortrading-2edd6fac689d
Comments.UserDisplayName: orde.r
Tags: <recurrent-neural-networks><long-short-term-memory><data-preprocessing><time-series><accuracy>
Title: Which preprocessing is the correct way to forecast time-series data using LSTM?
UserDisplayName: orde.r
ThreadText: # Which preprocessing is the correct way to forecast time-series data using LSTM?
  
  (Asked by: username_0 on 2023-06-08)
  
  I just started to study time-series forecasting using RNN.
  
  I have a few months of time series data that was an hour unit.
  The data is a kind of percentage value of my little experiment and I would like to forecast the future condition of this.
  Many tutorials and web info introduced direct training and forecasting the time series data without any data pre-processing.
  
  But for the RNN (or ML and DL), I think we should consider the data's condition that is stationary or not.
  
  My data is totally random condition which is stationary data (no seasonality, no trend).
  
  For example, the US stock prediction tutorial showed super great accuracy forecasting performance according to many LSTM tutorials.
  *[If this really works and is true, then all ML developers will be rich.]*
  
  And, Some of them didn't emphasize and note a kind of the data pre-processing such as non-stationary to stationary something like that.
  
  According to my short knowledge, I think the non-stationary data such as stock price (will have trend) should be converted as a stationary format through differencing or some other steps. and I think this is a correct prediction as a view of theoretical sense even if the accuracy is not high.
  
  So my point is, I'm a bit confused about whether that really is no need for any preprocessing to treat stationary or not.
  
  For my case, I applied differencing step ($t\_n - t\_{n-1}$)to my time-series data in order to remove the trend or some periodic situation.
  
  Is my understanding not correct?
  
  Why do time-series forecasting tutorials not introduced data stationarity?
  
  **Tags:** recurrent-neural-networks, long-short-term-memory, data-preprocessing, time-series, accuracy
  > **Comments:**
  > **username_1**: You stated that your data is 'totally random'. How do you know it is 'totally random'? Truly random time series data can not be forecasted. Stock price data is not random although it has some degree of randomness.  
  > **username_1**: You mentioned stock data and 'future condition'. That is very unclear. Are you trying to predict the future price of a stock given historical stock price data?  
  > **username_1**: You mentioned that a tutorial shows high forecasting accuracy. Please cite the tutorial. You pointed out that ML developers would be wealthy if high accuracy was possible. The fact is, practically all tutorials and papers do not report a valid time series forecasting metric! RMSE and other metrics that lose the directional accuracy of the forecast are useless for forecasting direction error. Another important fact is that LSTMs alone do not provide an accuracy suitable for forecasting stock prices for profit for long time regimes.  
  > **username_0**: Sorry for the unclear post. Actually, My data is a CPU Usage history from my lab top. This was a kind of practice work. According to my data, I found It seems no periodic situation or trend. So that is the reason why I said 'My data is totally random' but I think I misunderstood the random data such as stationary. the stock is one example to explain my question.  
  > **username_0**: Yes. Basically, I would like to predict future CPU Usage using CPU usage history as same as stock price prediction. In this step, not only 1 unit prediction a period prediction I want to do.  
  > **username_0**: I saw a few tutorial blogs and webpage that introduced 'LSTM for the time-series data'. most of the page is not written English (I'm sorry) and here is one of a page that I saw before. https://towardsdatascience.com/aifortrading-2edd6fac689d  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-06-09. Score: 3 | Accepted answer
  
  A standard method for pre-processing time series data for neural network architectures, such as an LSTM, is to normalize the data. Good tutorials will include this step. There are several variations to normalizing time series data. Jason Brownlee writes a lot of good stuff on machine learning. Please see his article on [How to Scale Data for Long Short-Term Memory Networks in Python](https://machinelearningmastery.com/how-to-scale-data-for-long-short-term-memory-networks-in-python/).
  
  Also, see the replies/answers to: <https://stackoverflow.com/questions/43467597/should-i-normalize-my-features-before-throwing-them-into-rnn>
  
  The following is a tutorial on predicting stock prices, employing normalization, using an LSTM: [Time-Series Forecasting: Predicting Stock Prices Using An LSTM Model](https://towardsdatascience.com/time-series-forecasting-predicting-stock-prices-using-an-lstm-model-d0056cd5e055)
  
  > **Comments:**
  > **username_0**: Thank you for your reply. It was a great helps to understand the LSTM using more reasonable data and model structure. I will carefully read the article that you recommended.  
  
  ---
  
  