Body: <p>I have a problem where the goal is for the agent to draw a single line between two points on a  <span class="math-container">$500 \times 500$</span> white image.</p>
  <p>I have built my DQN. For now, the output layer's size of the network is <span class="math-container">$[1, 500 * 500]$</span>. This way, when a Q value is given within a single step, it can be mapped to a single coordinate within the space of the image.</p>
  <p>So, with that, I'm able to get a starting point for the line. However, what that doesn't give me is the location of the second point for completion of the line.</p>
  <p>One thing I have tried is drawing a line for every two steps. However, this means that the state of the environment does not change for each step.</p>
  <p>Should the goal be to change the environment/state for each step or does this not matter? Maybe I have not found the ideal way of modelling the state and action spaces for this problem.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-19T19:39:55.367Z
Comments.Text: hi @junfanbl, what actions can the agent take?
Comments.UserDisplayName: mark mark
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-10T13:37:48.573Z
Comments.Text: I have edited this post to try to save it by asking the question "How to model the action and state space in this problem?". Make sure that you agree with these changes. If not, edit the post again.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><dqn><state-spaces><action-spaces><double-dqn>
Title: How should I model the state and action spaces for a problem where the goal is to draw a line between two points?
UserDisplayName: junfanbl
ThreadText: # How should I model the state and action spaces for a problem where the goal is to draw a line between two points?
  
  (Asked by: username_0 on 2021-02-06)
  
  I have a problem where the goal is for the agent to draw a single line between two points on a $500 \times 500$ white image.
  
  I have built my DQN. For now, the output layer's size of the network is $[1, 500 \* 500]$. This way, when a Q value is given within a single step, it can be mapped to a single coordinate within the space of the image.
  
  So, with that, I'm able to get a starting point for the line. However, what that doesn't give me is the location of the second point for completion of the line.
  
  One thing I have tried is drawing a line for every two steps. However, this means that the state of the environment does not change for each step.
  
  Should the goal be to change the environment/state for each step or does this not matter? Maybe I have not found the ideal way of modelling the state and action spaces for this problem.
  
  **Tags:** reinforcement-learning, dqn, state-spaces, action-spaces, double-dqn
  > **Comments:**
  > **username_1**: hi @junfanbl, what actions can the agent take?  
  > **username_2**: I have edited this post to try to save it by asking the question "How to model the action and state space in this problem?". Make sure that you agree with these changes. If not, edit the post again.  
  
  ---
  
  ## 0 answers
  
  