title: Q learning: Relearning after changing the environment
body: <p>I have implemented Q learning on a grid of size (n x n) with a single reward of 100 in the middle. The agent learns for 1000 epochs to reach the goal by the following agency: He chooses with probability 0.8 the move with the highest state-action-value and chooses a random move by 0.2. After a move the state-action value is updated by the Q learning rule.</p>
  
  <p>Now I did the following experiment: All fields next to the goal got a reward of -100 except the neighbour at the bottom. After learning for 1000 epochs the agent clearly avoids going the top way and arrives at the goal from the bottom most frequently.</p>
  
  <p>After learning set the reward of the bottom neighbour to -100 and the top neighbour back to 0 and start learning again for 1000 epochs while sticking with the state action value map.
  It's actually horrible! The agent needs very long to find the goal (on a 9x9 grid up to 3 minutes). After checking the paths I've seen that the agent spends a lot of time bouncing between two states like (0,0)->(1,0)->(0,0)->(1,0)...</p>
  
  <p>It is hard for me to imagine if this behaviour makes any sense. Has someone experience with a situation like this?</p>