Answers.Body: <p>Yes, DQN can do this, but if you want optimal behaviour, you will not be able to use a really simple representation of the state, with only the position of the agent tracked. In order to predict expected returns, and take an optimal path, the agent needs to be told somehow where these obstacles are.</p>
  <p>In this kind of scenario it is common to use some &quot;image&quot; of the grid that the agent is navigating, with walls, obstacles and the agent changing the value at the grid point they are occupying. Then the agent can use a convolutional neurtal network (CNN) to approximate Q values. You don't have to do this, any representation that tracks all changeable positions between episodes should work. However, the grid view of the environment is reasonably generic, and would cover many possible versions of the environment (e.g. you can change number of obstacles, add moving obstacles and other grid elements, all using a similar representation and starting agent model).</p>
  <p>The training session will need to include new random locations on each episode, so that the agent can learn to generalise how it should deal with them. It will take longer for the agent to learn this generalisation, than if it learned a static maze.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T07:55:36.797Z
Answers.Text: Thanks for your answer. I see what you mean. I can make all information on the grid world accessible by the agent except the obstacles, cause they are really generated in real time. Then maybe I have to look into solution for partially observable RL solutions?! At the moment I will try to train for many episodes so it can see all the possible locations of the obstacle but then the target locations (locations to be visited) also changes in each run. So it becomes way more complicated
Answers.UserDisplayName: Mamora
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T08:11:52.720Z
Answers.Text: @Mamora if the agent is allowed to see the obstacles then you can put them on a grid. This is a lot like the DQN Atari agent. If the obstacles have to be invisible to the agent and it needs to learn their position in real time by bumping into them, then I might have some suggestions - ask a new question and give more details of your problem
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T08:12:03.550Z
Answers.Text: @Mamora if the agent is allowed to see the obstacles then you can put them on a grid. This is a lot like the DQN Atari agent. If the obstacles have to be invisible to the agent and it needs to learn their position in real time by bumping into them, then I might have some suggestions - ask a new question and give more details of your problem
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T08:13:45.530Z
Answers.Text: @Niel Slater, the obstacle is only visible when the agent is close to it. It can see one step ahead. Thanks, I will ask a new question then :)
Answers.UserDisplayName: Mamora
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T09:26:55.763Z
Answers.Text: @Niel Slater this is the new question with more details about the problem: https://ai.stackexchange.com/q/41112/73668
Answers.UserDisplayName: Mamora
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-03T07:36:55.143Z
Answers.LastActivityDate: 2023-07-03T07:36:55.143Z
Answers.UserDisplayName: Neil Slater
Body: <p>Can an agent trained with a DQN algorithm in a grid world, avoid obstacles (randomly appearing during the run time) and still find the optimal path to finish a task? The agent is supposed to visit specific locations (which is also different each time) and it may encounters obstacles. The goal is to visit those locations with the shortest path possible.</p>
  
Tags: <reinforcement-learning><dqn><robotics>
Title: Can DQN find optimal path while avoiding random obstacles?
UserDisplayName: Mamora
ThreadText: # Can DQN find optimal path while avoiding random obstacles?
  
  (Asked by: username_0 on 2023-07-03)
  
  Can an agent trained with a DQN algorithm in a grid world, avoid obstacles (randomly appearing during the run time) and still find the optimal path to finish a task? The agent is supposed to visit specific locations (which is also different each time) and it may encounters obstacles. The goal is to visit those locations with the shortest path possible.
  
  **Tags:** reinforcement-learning, dqn, robotics
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-07-03. Score: 2 | Accepted answer
  
  Yes, DQN can do this, but if you want optimal behaviour, you will not be able to use a really simple representation of the state, with only the position of the agent tracked. In order to predict expected returns, and take an optimal path, the agent needs to be told somehow where these obstacles are.
  
  In this kind of scenario it is common to use some "image" of the grid that the agent is navigating, with walls, obstacles and the agent changing the value at the grid point they are occupying. Then the agent can use a convolutional neurtal network (CNN) to approximate Q values. You don't have to do this, any representation that tracks all changeable positions between episodes should work. However, the grid view of the environment is reasonably generic, and would cover many possible versions of the environment (e.g. you can change number of obstacles, add moving obstacles and other grid elements, all using a similar representation and starting agent model).
  
  The training session will need to include new random locations on each episode, so that the agent can learn to generalise how it should deal with them. It will take longer for the agent to learn this generalisation, than if it learned a static maze.
  
  > **Comments:**
  > **username_0**: Thanks for your answer. I see what you mean. I can make all information on the grid world accessible by the agent except the obstacles, cause they are really generated in real time. Then maybe I have to look into solution for partially observable RL solutions?! At the moment I will try to train for many episodes so it can see all the possible locations of the obstacle but then the target locations (locations to be visited) also changes in each run. So it becomes way more complicated  
  > **username_1**: @username_0 if the agent is allowed to see the obstacles then you can put them on a grid. This is a lot like the DQN Atari agent. If the obstacles have to be invisible to the agent and it needs to learn their position in real time by bumping into them, then I might have some suggestions - ask a new question and give more details of your problem  
  > **username_1**: @username_0 if the agent is allowed to see the obstacles then you can put them on a grid. This is a lot like the DQN Atari agent. If the obstacles have to be invisible to the agent and it needs to learn their position in real time by bumping into them, then I might have some suggestions - ask a new question and give more details of your problem  
  > **username_0**: @Niel username_1, the obstacle is only visible when the agent is close to it. It can see one step ahead. Thanks, I will ask a new question then :)  
  > **username_0**: @Niel username_1 this is the new question with more details about the problem: https://ai.stackexchange.com/q/41112/73668  
  
  ---
  
  