Answers.Body: <p>Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset. </p>
  
  <p>Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be numerically unstable and statistical inefficient (so convergence might be slow). See <a href="https://arxiv.org/pdf/1505.02417.pdf" rel="nofollow noreferrer">Towards stability and optimality in stochastic gradient descent</a> for more details and a solution (AI-SGD).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-07T13:51:30.053Z
Answers.Text: This is an addition to [John Doucette's](https://ai.stackexchange.com/a/12180/2444) answer.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-07T13:50:51.917Z
Answers.LastActivityDate: 2019-05-07T13:50:51.917Z
Answers.UserDisplayName: nbro
Answers.Body: <p>ANNs &amp; RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of <a href="https://en.wikipedia.org/wiki/Hyperparameter_optimization" rel="nofollow noreferrer">hyper-parameter optimization</a>. Because neural networks are so flexible, it is not clear, at the outset, which <a href="https://cdn-images-1.medium.com/max/2400/1*cuTSPlTq0a_327iTPJyD-Q.png" rel="nofollow noreferrer">arrangement of neurons</a> will be most effective to solve a given problem. It is also not clear how fast the network should learn from new signals, what sorts of activation functions to use in the different layers of the network, and which of <a href="https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/" rel="nofollow noreferrer">several possible regularization methods</a> might be best. Making these decisions well requires either years of practice and experience, or a lot of trial and error (or, maybe both!).</p>
  
  <p>In contrast, a regression-based method like ARMA will typically have just a couple of simple hyperparameters, each of which has a clear, intuitive, meaning. This means that an untrained practitioner can probably get an ARMA result that is close to the result of a trained practitioner using ARMA. </p>
  
  <p>Essentially: neural networks are brittle and sensitive to the choice of hyper-parameters, while regression generally is not.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-07T12:44:46.773Z
Answers.LastActivityDate: 2019-05-07T12:44:46.773Z
Answers.UserDisplayName: John Doucette
Body: <p>There are different kinds of machine learning algorithms, both univariate and multivariate, that are used for time series forecasting: for example ARIMA, VAR or AR.</p>
  
  <p>Why is it harder (compared to classical models like ARIMA) to achieve good results using neural network based algorithms (like ANN and RNN) for multi step time series forecasting?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-07T13:35:10.533Z
Comments.Text: Have look at this blog post: [https://machinelearningmastery.com/findings-comparing-classical-and-machine-learning-methods-for-time-series-forecasting/](https://machinelearningmastery.com/findings-comparing-classical-and-machine-learning-methods-for-time-series-forecasting/). See also these two related questions: [https://stats.stackexchange.com/q/223209/82135](https://stats.stackexchange.com/q/223209/82135) and [https://datascience.stackexchange.com/q/12721/10640](https://datascience.stackexchange.com/q/12721/10640).
Comments.UserDisplayName: nbro
Tags: <deep-learning><prediction><comparison><forecasting><time-series>
Title: Why is it harder to achieve good results using neural network based algorithms for multi step time series forecasting?
UserDisplayName: Majo_Jose
ThreadText: # Why is it harder to achieve good results using neural network based algorithms for multi step time series forecasting?
  
  (Asked by: username_0 on 2019-05-06)
  
  There are different kinds of machine learning algorithms, both univariate and multivariate, that are used for time series forecasting: for example ARIMA, VAR or AR.
  
  Why is it harder (compared to classical models like ARIMA) to achieve good results using neural network based algorithms (like ANN and RNN) for multi step time series forecasting?
  
  **Tags:** deep-learning, prediction, comparison, forecasting, time-series
  
  > **Comments:**
  > **username_1**: Have look at this blog post: [https://machinelearningmastery.com/findings-comparing-classical-and-machine-learning-methods-for-time-series-forecasting/](https://machinelearningmastery.com/findings-comparing-classical-and-machine-learning-methods-for-time-series-forecasting/). See also these two related questions: [https://stats.stackexchange.com/q/223209/82135](https://stats.stackexchange.com/q/223209/82135) and [https://datascience.stackexchange.com/q/12721/10640](https://datascience.stackexchange.com/q/12721/10640).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-05-07. Score: 3 | Accepted answer
  
  ANNs & RNNs can be used to create some great models in many different domains, including time-series forecasting. However, across all of these domains, they suffer from the problem of [hyper-parameter optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization). Because neural networks are so flexible, it is not clear, at the outset, which [arrangement of neurons](https://cdn-images-1.medium.com/max/2400/1*cuTSPlTq0a_327iTPJyD-Q.png) will be most effective to solve a given problem. It is also not clear how fast the network should learn from new signals, what sorts of activation functions to use in the different layers of the network, and which of [several possible regularization methods](https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/) might be best. Making these decisions well requires either years of practice and experience, or a lot of trial and error (or, maybe both!).
  
  In contrast, a regression-based method like ARMA will typically have just a couple of simple hyperparameters, each of which has a clear, intuitive, meaning. This means that an untrained practitioner can probably get an ARMA result that is close to the result of a trained practitioner using ARMA.
  
  Essentially: neural networks are brittle and sensitive to the choice of hyper-parameters, while regression generally is not.
  
  
  ---
  
  ### Answer by username_1 on 2019-05-07. Score: 2
  
  Given the (usual) higher architectural complexity of ML models compared to more classical forecasting models, ML models might also require more data, otherwise they might just overfit the training dataset.
  
  Furthermore, online learning (or training) of a neural network using stochastic gradient descent (that is, one example at a time) might also be numerically unstable and statistical inefficient (so convergence might be slow). See [Towards stability and optimality in stochastic gradient descent](https://arxiv.org/pdf/1505.02417.pdf) for more details and a solution (AI-SGD).
  
  > **Comments:**
  > **username_1**: This is an addition to [username_2 username_2's](https://ai.stackexchange.com/a/12180/2444) answer.  
  
  ---
  
  