Answers.Body: <p>This is just a case of supervised learning. You are trying to predict <span class="math-container">$s_{t+1}$</span> given <span class="math-container">$s_t$</span> and <span class="math-container">$a_t$</span>, so the answer to your question depends on how complex your state dynamics are.</p>
  <p>For example, if the state space is really complex, e.g. if your state space is an image and you want to predict the next image given the current image and an action, then linear methods are unlikely to work well.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-14T15:14:42.083Z
Answers.Text: Yes, state space will be key. [This work](https://arxiv.org/abs/2002.09093) uses a $32 * 32$ image on a linear model. Would that be termed as complex?
Answers.UserDisplayName: mugoh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-14T15:46:36.877Z
Answers.Text: I'm not sure, I'm not an image expert. In [this paper](https://arxiv.org/pdf/1912.01603) they learn state dynamics in a latent space -- the raw state space is an image -- and they use deep nets for this, so I would assume you need something more complex than a linear model to predict images.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-14T13:21:06.587Z
Answers.LastActivityDate: 2020-09-14T13:21:06.587Z
Answers.UserDisplayName: David
Body: <p>Model based RL attempts to learn a function <span class="math-container">$f(s_{t+1}|s_t, a_t)$</span> representing the environment transitions, otherwise known as a model of the system. I see linear functions are still being used in model-based RL <a href="https://arxiv.org/abs/2002.09093" rel="nofollow noreferrer">such as in robotic manipulation</a> to learn system dynamics, and can work effectively well. (Here, I mean in learning the model, not as an optimization method for the controller selecting the best actions).</p>
  <p>In model-based RL, are there situations where a learning a linear model such as using a <a href="https://en.wikipedia.org/wiki/Lyapunov_function" rel="nofollow noreferrer">Lyapunov function</a> would be better suited than using a neural network, or are the examples of problems framed to use linear models when addressing them using model-based RL?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-26T06:28:19.127Z
Comments.Text: This might be helpful: "Linear approximations of non-linear physical systems prove accurate and are well accepted in many fields. For the *comprehensive* study, see [Linear models of nonlinear systems](https://www.diva-portal.org/smash/get/diva2:21266/FULLTEXT01.pdf)"
Comments.UserDisplayName: mugoh
Tags: <reinforcement-learning><control-problem><model-based-methods><linear-programming>
Title: Are linear approximators better suited to some tasks compared to complex neural net functions?
UserDisplayName: mugoh
ThreadText: # Are linear approximators better suited to some tasks compared to complex neural net functions?
  
  (Asked by: username_0 on 2020-09-14)
  
  Model based RL attempts to learn a function $f(s\_{t+1}|s\_t, a\_t)$ representing the environment transitions, otherwise known as a model of the system. I see linear functions are still being used in model-based RL [such as in robotic manipulation](https://arxiv.org/abs/2002.09093) to learn system dynamics, and can work effectively well. (Here, I mean in learning the model, not as an optimization method for the controller selecting the best actions).
  
  In model-based RL, are there situations where a learning a linear model such as using a [Lyapunov function](https://en.wikipedia.org/wiki/Lyapunov_function) would be better suited than using a neural network, or are the examples of problems framed to use linear models when addressing them using model-based RL?
  
  **Tags:** reinforcement-learning, control-problem, model-based-methods, linear-programming
  
  > **Comments:**
  > **username_0**: This might be helpful: "Linear approximations of non-linear physical systems prove accurate and are well accepted in many fields. For the *comprehensive* study, see [Linear models of nonlinear systems](https://www.diva-portal.org/smash/get/diva2:21266/FULLTEXT01.pdf)"  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-09-14. Score: 1
  
  This is just a case of supervised learning. You are trying to predict $s\_{t+1}$ given $s\_t$ and $a\_t$, so the answer to your question depends on how complex your state dynamics are.
  
  For example, if the state space is really complex, e.g. if your state space is an image and you want to predict the next image given the current image and an action, then linear methods are unlikely to work well.
  
  > **Comments:**
  > **username_0**: Yes, state space will be key. [This work](https://arxiv.org/abs/2002.09093) uses a $32 * 32$ image on a linear model. Would that be termed as complex?  
  > **username_1**: I'm not sure, I'm not an image expert. In [this paper](https://arxiv.org/pdf/1912.01603) they learn state dynamics in a latent space -- the raw state space is an image -- and they use deep nets for this, so I would assume you need something more complex than a linear model to predict images.  
  
  ---
  
  