Body: <p>Does it make sense to use Reinforcement Learning methods in an environment that does not have trajectories?</p>
  
  <p>I have a lot of states and actions in my environment. However, there are no trajectories.</p>
  
  <p>If the agent takes action <span class="math-container">$a$</span> in the state <span class="math-container">$s$</span>, it reaches <span class="math-container">$s'$</span>. In <span class="math-container">$s'$</span>, it takes action <span class="math-container">$a'$</span> and reaches state <span class="math-container">$s''$</span>. However, if it takes the reverse order of actions <span class="math-container">$a'$</span> and then <span class="math-container">$a$</span>, it would reach the same state <span class="math-container">$s''$</span>.</p>
  
  <p>How reinforcement learning methods handle with this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-17T18:03:37.990Z
Comments.Text: There is still a "trajectory" in your example, and in fact it also fits to a simple grid  world where $a$ is "go north", $a'$ is "go east" and $s''$ is the postion North East of  $s$. But that doesn't seem to match your concerns? I would guess that you mean that choice of $a$ has no impact on the transition $s \rightarrow s'$? Does $a$ influence anything else - what does the reward depend upon? Is the state progression deterministic in any useful way i.e. $s \rightarrow s'$ has a probability distribution that is not simply equiprobable random sampling from all possible states?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-18T10:28:30.900Z
Comments.Text: @NeilSlater Thanks for your answer.
  I will make it more clear.
  I have a set of actions that affect my environment, and I want to find them.
  Executing these actions from any state affect the environment in the same way.
  About the reward, I get a non-zero reward from the environment only at the end of the trajectory.
  This reward is vary from state to state (most of the states are 0).
Comments.UserDisplayName: NNMM
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-18T14:01:10.243Z
Comments.Text: @NoamMoscovich Just edit your post to clarify the question and problem! This should also attract other people, because when you edit a post it will pop up again in the front page.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-16T21:03:25.543Z
Comments.Text: Please, I know this post is old, but try to clarify what your problem really is. As Neil said, it seems that there's a trajectory in your case.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning>
Title: Reinforcement learning without trajectories
UserDisplayName: NNMM
ThreadText: # Reinforcement learning without trajectories
  
  (Asked by: username_0 on 2019-12-17)
  
  Does it make sense to use Reinforcement Learning methods in an environment that does not have trajectories?
  
  I have a lot of states and actions in my environment. However, there are no trajectories.
  
  If the agent takes action $a$ in the state $s$, it reaches $s'$. In $s'$, it takes action $a'$ and reaches state $s''$. However, if it takes the reverse order of actions $a'$ and then $a$, it would reach the same state $s''$.
  
  How reinforcement learning methods handle with this?
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: There is still a "trajectory" in your example, and in fact it also fits to a simple grid  world where $a$ is "go north", $a'$ is "go east" and $s''$ is the postion North East of  $s$. But that doesn't seem to match your concerns? I would guess that you mean that choice of $a$ has no impact on the transition $s \rightarrow s'$? Does $a$ influence anything else - what does the reward depend upon? Is the state progression deterministic in any useful way i.e. $s \rightarrow s'$ has a probability distribution that is not simply equiprobable random sampling from all possible states?  
  > **username_0**: @username_1 Thanks for your answer. I will make it more clear. I have a set of actions that affect my environment, and I want to find them. Executing these actions from any state affect the environment in the same way. About the reward, I get a non-zero reward from the environment only at the end of the trajectory. This reward is vary from state to state (most of the states are 0).  
  > **username_2**: @NoamMoscovich Just edit your post to clarify the question and problem! This should also attract other people, because when you edit a post it will pop up again in the front page.  
  > **username_2**: Please, I know this post is old, but try to clarify what your problem really is. As username_1 said, it seems that there's a trajectory in your case.  
  
  ---
  
  ## 0 answers
  
  