Answers.Body: <p>I have encountered similar problem while trying to predict forex prices. I understand it this way:</p>
  
  <blockquote>
    <p>The data based on which you try to model price differences are so poor that the lowest error is achieved by zeroing the predicted values</p>
  </blockquote>
  
  <p>In other words, due to poor "correlation" in data, zero values are the best solution.</p>
  
  <p>My advice would be to look for more correlated data to be used.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-03-05T18:44:27.790Z
Answers.LastActivityDate: 2019-03-06T12:27:39.840Z
Answers.LastEditDate: 2019-03-06T12:27:39.840Z
Answers.UserDisplayName: GKozinski
Body: <p>With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well as the features. In other words, at time <code>t+1</code>, we predict the price difference and the features for time <code>t+2</code>. We use the predicted features from time <code>t+1</code> to predict the price at time <code>t+2</code>. </p>
  
  <p>We got very excited, because we thought getting good results with the following graph</p>
  
  <p><a href="https://i.sstatic.net/xSVd8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xSVd8.png" alt="enter image description here"></a></p>
  
  <p>We got problems in production and we wasn't known the problem till we plot the price difference.</p>
  
  <p><a href="https://i.sstatic.net/PZu4F.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PZu4F.png" alt="enter image description here"></a></p>
  
  <p>Here is the content of the config file</p>
  
  <pre><code>{
      "data": {
          "sequence_length":30,
          "train_test_split": 0.85,
          "normalise": false,
          "num_steps": 5
      },
      "training": {
          "epochs":200,
          "batch_size": 64
      },
      "model": {
          "loss": "mse",
          "optimizer": "adam",
          "layers": [
              {
                  "type": "lstm",
                  "neurons": 51,
                  "input_timesteps": 30,
                  "input_dim": 101,
                  "return_seq": true,
                  "activation": "relu"
              },
              {
                  "type": "dropout",
                  "rate": 0.1
              },
              {
                  "type": "lstm",
                  "neurons": 51,
                  "activation": "relu",
                  "return_seq": false
              },
              {
                  "type": "dropout",
                  "rate": 0.1
              },
              {
                  "type": "dense",
                  "neurons": 101,
                  "activation": "relu"
              },
              {
                  "type": "dense",
                  "neurons": 101,
                  "activation": "linear"
              }
          ]
      }
  }
  </code></pre>
  
  <p>Prices don't change very fast. Therefore, the next price is almost always very close to the last price. In other words, <code>P_{t+1} - P_{t</code>} is very often close to zero or zero directly. If there is too many zeros then the network will only recognize the zeros. The model has picked up on that.</p>
  
  <p>I guess the model learned almost nothing except the very simple relationship that the next price is close to the last price. There is not necessarily anything wrong with the model. Predicting stock prices should be a very hard problem. </p>
  
  <p>So a straightforward improvement should be of taking the features as a whole instead of their difference. </p>
  
  <p>I want to keep working with price difference instead of the price in itself because we are making the series potential more stationary.</p>
  
  <p>What might be a good solution to deal with the repetitive zeros related to our "price difference" problem? Does applying the log-return is a better idea than applying price differences?</p>
  
  <p>Does a zero inflated estimators is a good idea? First predict whether it's gonna be a zero. If not predict the value. <a href="https://gist.github.com/fonnesbeck/874808" rel="nofollow noreferrer">https://gist.github.com/fonnesbeck/874808</a> ?</p>
  
Tags: <neural-networks><machine-learning><deep-learning><long-short-term-memory>
Title: Price difference predictions curve almost vanished
UserDisplayName: user1050421
ThreadText: # Price difference predictions curve almost vanished
  
  (Asked by: username_0 on 2018-11-05)
  
  With a team, we are studying how it is possible to predict the price movement with high-frequency. Instead of predicting the price directly, we have decided to try predicting price difference as well as the features. In other words, at time t+1, we predict the price difference and the features for time t+2. We use the predicted features from time t+1 to predict the price at time t+2. 
  We got very excited, because we thought getting good results with the following graph
  
  We got problems in production and we wasn't known the problem till we plot the price difference.
  
  Here is the content of the config file
  {
      "data": {
          "sequence_length":30,
          "train_test_split": 0.85,
          "normalise": false,
          "num_steps": 5
      },
      "training": {
          "epochs":200,
          "batch_size": 64
      },
      "model": {
          "loss": "mse",
          "optimizer": "adam",
          "layers": [
              {
                  "type": "lstm",
                  "neurons": 51,
                  "input_timesteps": 30,
                  "input_dim": 101,
                  "return_seq": true,
                  "activation": "relu"
              },
              {
                  "type": "dropout",
                  "rate": 0.1
              },
              {
                  "type": "lstm",
                  "neurons": 51,
                  "activation": "relu",
                  "return_seq": false
              },
              {
                  "type": "dropout",
                  "rate": 0.1
              },
              {
                  "type": "dense",
                  "neurons": 101,
                  "activation": "relu"
              },
              {
                  "type": "dense",
                  "neurons": 101,
                  "activation": "linear"
              }
          ]
      }
  }
  
  Prices don't change very fast. Therefore, the next price is almost always very close to the last price. In other words, P_{t+1} - P_{t} is very often close to zero or zero directly. If there is too many zeros then the network will only recognize the zeros. The model has picked up on that.
  I guess the model learned almost nothing except the very simple relationship that the next price is close to the last price. There is not necessarily anything wrong with the model. Predicting stock prices should be a very hard problem. 
  So a straightforward improvement should be of taking the features as a whole instead of their difference. 
  I want to keep working with price difference instead of the price in itself because we are making the series potential more stationary.
  What might be a good solution to deal with the repetitive zeros related to our "price difference" problem? Does applying the log-return is a better idea than applying price differences?
  Does a zero inflated estimators is a good idea? First predict whether it's gonna be a zero. If not predict the value. https://gist.github.com/fonnesbeck/874808 ?
  
  **Tags:** neural-networks, machine-learning, deep-learning, long-short-term-memory
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-03-05. Score: 0
  
  I have encountered similar problem while trying to predict forex prices. I understand it this way:
  
  > The data based on which you try to model price differences are so poor that the lowest error is achieved by zeroing the predicted values
  
  In other words, due to poor "correlation" in data, zero values are the best solution.
  
  My advice would be to look for more correlated data to be used.
  
  
  ---
  
  