Answers.Body: <p>The simplest thing you can do is to add data regarding the target pose to the state vector. This will allow any generalisations that the agent learns that apply to similar poses to be used directly.</p>
  <p>Clearly in normal use, where the target pose remains fixed during the episode, then that part of the state information will not change either during the episode. You will also need to train with a large variety of target poses - so training will take longer.</p>
  <p>Multi-Objective Deep Reinforcement Learning is slightly different in that it attempts to resolve prioritising between multiple sub-goals. It would also be a more complex solution, whilst augmenting the state vector should allow you to continue using a very similar DQN set up as you have already.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-23T14:51:22.637Z
Answers.Text: Thanks for your explanation. So the state vector would maybe look like `(...,current_pose, target_pose)`, where `target_pose` changes after every episode? I already included those information in my current implementation. I think my mistake was that I changed the `target_pose` after the agent found the previous one during my first tries. Very appreciated your help.
Answers.UserDisplayName: PhilG
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-23T16:34:05.453Z
Answers.Text: Yes. If you can, then you should randomise `target_episode` according to expected distribution in production for each episode whilst training. If the initial pose can vary then you should randomise that too - again the ideal is to match distribution it would work with in production.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-23T13:46:58.957Z
Answers.LastActivityDate: 2020-06-23T13:46:58.957Z
Answers.UserDisplayName: Neil Slater
Body: <p>I created a virtual 2D environment where an agent aims to find a correct pose corresponding to a target image. I implemented a DQN to solve this task. When the goal is fixed, e.g. the aim is to find the pose for position (1,1), the agent is successful. I would now like to train an agent to find the correct pose while the goal pose changes after every episode. My research pointed me to the term &quot;Multi-Objective Deep Reinforcement Learning&quot;. As far as I understood, the aim here is to train one or multiple agents to achieve a policy approximation that fits all goals.
  Am I on the right track or how should I deal with different goal states?</p>
  
Tags: <reinforcement-learning><dqn>
Title: How to handle changing goals in a DQN?
UserDisplayName: PhilG
ThreadText: # How to handle changing goals in a DQN?
  
  (Asked by: username_0 on 2020-06-23)
  
  I created a virtual 2D environment where an agent aims to find a correct pose corresponding to a target image. I implemented a DQN to solve this task. When the goal is fixed, e.g. the aim is to find the pose for position (1,1), the agent is successful. I would now like to train an agent to find the correct pose while the goal pose changes after every episode. My research pointed me to the term "Multi-Objective Deep Reinforcement Learning". As far as I understood, the aim here is to train one or multiple agents to achieve a policy approximation that fits all goals.
  Am I on the right track or how should I deal with different goal states?
  
  **Tags:** reinforcement-learning, dqn
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-23. Score: 1 | Accepted answer
  
  The simplest thing you can do is to add data regarding the target pose to the state vector. This will allow any generalisations that the agent learns that apply to similar poses to be used directly.
  
  Clearly in normal use, where the target pose remains fixed during the episode, then that part of the state information will not change either during the episode. You will also need to train with a large variety of target poses - so training will take longer.
  
  Multi-Objective Deep Reinforcement Learning is slightly different in that it attempts to resolve prioritising between multiple sub-goals. It would also be a more complex solution, whilst augmenting the state vector should allow you to continue using a very similar DQN set up as you have already.
  
  > **Comments:**
  > **username_0**: Thanks for your explanation. So the state vector would maybe look like `(...,current_pose, target_pose)`, where `target_pose` changes after every episode? I already included those information in my current implementation. I think my mistake was that I changed the `target_pose` after the agent found the previous one during my first tries. Very appreciated your help.  
  > **username_1**: Yes. If you can, then you should randomise `target_episode` according to expected distribution in production for each episode whilst training. If the initial pose can vary then you should randomise that too - again the ideal is to match distribution it would work with in production.  
  
  ---
  
  