Answers.Body: <p>Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to. </p>
  
  <p>For example, in your specific case, you could model a bubble bursting as perhaps a negative exponential drop-off or something of the sort. This could include machine learning too. You could gather historical drops in stock market data, and use some sort of regression (Bayesian would probably work well) to estimate the best function to use as an indicator to whether a steep drop has occurred. If so, then use your neural network specifically to classify the fate of the stock. I would think you would have more success following a specialised route such as this rather than trying to train a network on general trends in the market.</p>
  
  <p>In terms of the structure of your neural network, you may want to consider a convolutional neural network (CNN) instead of a recurrent neural network (RNN). RNNs assume the current point in your time-series depends on all previous points, from the beginning of your data. I wouldn't think this would hold true in general for the stock market. The filters a CNN learns are suited to learning to extract certain features and the CNN will apply the filters to specific portions of the data, in the way it considers optimal. They are both nonlinear models, but the CNN will be less computationally costly to train. You could also try a gradient-boosting regression approach instead of a neural network. That being said, something like an LSTM RNN (long short-term memory) won't necessarily be bad - just my two cents.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-10T13:19:50.683Z
Answers.LastActivityDate: 2020-03-10T13:19:50.683Z
Answers.UserDisplayName: Archie Shahidullah
Answers.Body: <p>Tbh I think stock prices are essentially impossible to predict as you're not taking into account the data from outside the stock market.</p>
  
  <p>I'd argue any successful model would need to be trained on news, consumer sentiment, etc etc.</p>
  
  <p>The only ones which maybe work are HFT. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-07T20:43:36.410Z
Answers.Text: that is not true, stock prices can be predicted, this was already proven by some systems that already work. They can't be predicted always, but at certain moments in time, they are very predictable. The "unpredictability" of stocks is promoted by academia, but academia doesn't have very good domain knowledge of financial industry. Otherwise we wouldn't have hedge funds making profits year after year and hedge funds use mathematical models.
Answers.UserDisplayName: Nulik
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-07T19:13:14.080Z
Answers.LastActivityDate: 2020-03-07T19:13:14.080Z
Answers.UserDisplayName: FourierFlux
Body: <p>I am trying to predict Forex time series. The nature of the market is that 80% of the time the price can not be predicted, but in 20% of the time it can be. For example, if the price drops down very deep, there is 99% probability that there will be a recovery process, and this is what I want to predict.</p>
  
  <p>So , how do I train a feed-forward network the way it would only predict those cases that have 99% of certainty to take place and, for the rest of the cases it would output "unpredictable" status?</p>
  
  <p>Imagine that my data set has 24 hours of continuous price data as input (as 1 minute samples), and then as output I want the network to predict 1 hour of future price data. The only restriction I need to implement is that if the network is not "sure" that the price is predictable, it would outupt 0s. So, how do I implement safety in predictions the network is outputting?</p>
  
  <p>It seems that my problem is similar to Google Compose, where it predicts the next word as you are typing , for example, if you type "thank you", it would add " very much" and this would be like 95% correct. I want the same, but it is just that my problem has too much complexity. Google uses RNNs, so maybe I should try a deep network of many layers of RNNs?</p>
  
Tags: <machine-learning><time-series>
Title: How to predict time series with accuracy?
UserDisplayName: Nulik
ThreadText: # How to predict time series with accuracy?
  
  (Asked by: username_0 on 2020-03-07)
  
  I am trying to predict Forex time series. The nature of the market is that 80% of the time the price can not be predicted, but in 20% of the time it can be. For example, if the price drops down very deep, there is 99% probability that there will be a recovery process, and this is what I want to predict.
  
  So , how do I train a feed-forward network the way it would only predict those cases that have 99% of certainty to take place and, for the rest of the cases it would output "unpredictable" status?
  
  Imagine that my data set has 24 hours of continuous price data as input (as 1 minute samples), and then as output I want the network to predict 1 hour of future price data. The only restriction I need to implement is that if the network is not "sure" that the price is predictable, it would outupt 0s. So, how do I implement safety in predictions the network is outputting?
  
  It seems that my problem is similar to Google Compose, where it predicts the next word as you are typing , for example, if you type "thank you", it would add " very much" and this would be like 95% correct. I want the same, but it is just that my problem has too much complexity. Google uses RNNs, so maybe I should try a deep network of many layers of RNNs?
  
  **Tags:** machine-learning, time-series
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2020-03-10. Score: 2
  
  Things like this a really hot topic in research right now, and it's very difficult to get high accuracy on a chaotic system like the stock market. That being said, I would probably recommend preprocessing your data rather than having your primary neural network decide what to accept and what not to.
  
  For example, in your specific case, you could model a bubble bursting as perhaps a negative exponential drop-off or something of the sort. This could include machine learning too. You could gather historical drops in stock market data, and use some sort of regression (Bayesian would probably work well) to estimate the best function to use as an indicator to whether a steep drop has occurred. If so, then use your neural network specifically to classify the fate of the stock. I would think you would have more success following a specialised route such as this rather than trying to train a network on general trends in the market.
  
  In terms of the structure of your neural network, you may want to consider a convolutional neural network (CNN) instead of a recurrent neural network (RNN). RNNs assume the current point in your time-series depends on all previous points, from the beginning of your data. I wouldn't think this would hold true in general for the stock market. The filters a CNN learns are suited to learning to extract certain features and the CNN will apply the filters to specific portions of the data, in the way it considers optimal. They are both nonlinear models, but the CNN will be less computationally costly to train. You could also try a gradient-boosting regression approach instead of a neural network. That being said, something like an LSTM RNN (long short-term memory) won't necessarily be bad - just my two cents.
  
  
  ---
  
  ### Answer by username_2 on 2020-03-07. Score: 0
  
  Tbh I think stock prices are essentially impossible to predict as you're not taking into account the data from outside the stock market.
  
  I'd argue any successful model would need to be trained on news, consumer sentiment, etc etc.
  
  The only ones which maybe work are HFT.
  
  > **Comments:**
  > **username_0**: that is not true, stock prices can be predicted, this was already proven by some systems that already work. They can't be predicted always, but at certain moments in time, they are very predictable. The "unpredictability" of stocks is promoted by academia, but academia doesn't have very good domain knowledge of financial industry. Otherwise we wouldn't have hedge funds making profits year after year and hedge funds use mathematical models.  
  
  ---
  
  