Answers.Body: <p>Yes, there are RL algorithms (dreamer v1,2,3) that predict world model to simulate what (h_t+1) given its sampled action and previous state (h_t). It runs simulated world model during critic-actor so it can be much more data efficient. <a href="https://arxiv.org/abs/2301.04104v1" rel="nofollow noreferrer">https://arxiv.org/abs/2301.04104v1</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T11:24:39.433Z
Answers.Text: You can use mathjax on this site.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-02T09:31:53.753Z
Answers.Text: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-01-31T10:13:18.833Z
Answers.LastActivityDate: 2023-01-31T12:42:03.767Z
Answers.LastEditDate: 2023-01-31T12:42:03.767Z
Answers.UserDisplayName: ipoppo
Body: <p>I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and Police Gradient methods for that implementation. I will try to explain the sequential logic of the task to better understand...
  From the current observation (o_t), the agent takes actions reaching a new state (s_t+1). This new state is indeed a function of the previous obersvation (o_t) and the actions (a_t) taken by the agent. However, this next state (s_t+1) is not the next observation that the agent will take into account to take new actions, being only useful for calculating the reward in that iteration for the policy optimization. My question consists of the possibility that RL can be applied to this type of problem, where the state reached by the agent's actions in a given instant of time is not my next observation that the agent will take into account to take the next actions.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-25T11:00:49.537Z
Comments.Text: Not an expert on RL myself yet, but it seems you are describing a POMDP = Partially Observable MDP (as opposed to a vanilla MDP) which is used a lot in RL. So yes :)
Comments.UserDisplayName: Felix Goldberg
Tags: <reinforcement-learning><deep-learning><deep-rl><continuous-action-spaces><continuous-state-spaces>
Title: RL - Can RL be applied to problems where the next state is not the next observation?
UserDisplayName: MaarcosNascimen
ThreadText: # RL - Can RL be applied to problems where the next state is not the next observation?
  
  (Asked by: username_0 on 2022-11-29)
  
  I'm quite new on the study of reinforcement learning, and Im working on a communication problem with continuous large actions range for my final graduation work. I'm trying to use Gaussian Policy and Police Gradient methods for that implementation. I will try to explain the sequential logic of the task to better understand...
  From the current observation (o\_t), the agent takes actions reaching a new state (s\_t+1). This new state is indeed a function of the previous obersvation (o\_t) and the actions (a\_t) taken by the agent. However, this next state (s\_t+1) is not the next observation that the agent will take into account to take new actions, being only useful for calculating the reward in that iteration for the policy optimization. My question consists of the possibility that RL can be applied to this type of problem, where the state reached by the agent's actions in a given instant of time is not my next observation that the agent will take into account to take the next actions.
  
  **Tags:** reinforcement-learning, deep-learning, deep-rl, continuous-action-spaces, continuous-state-spaces
  
  > **Comments:**
  > **username_1**: Not an expert on RL myself yet, but it seems you are describing a POMDP = Partially Observable MDP (as opposed to a vanilla MDP) which is used a lot in RL. So yes :)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-01-31. Score: 0
  
  Yes, there are RL algorithms (dreamer v1,2,3) that predict world model to simulate what (h\_t+1) given its sampled action and previous state (h\_t). It runs simulated world model during critic-actor so it can be much more data efficient. <https://arxiv.org/abs/2301.04104v1>
  
  > **Comments:**
  > **username_3**: You can use mathjax on this site.  
  > **username_4**: As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  