Body: <p>I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.</p>
  
  <p>I thought, like the Deepmind "Playing Atari with Deep Reinforcement Learning" Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.</p>
  
  <p>Unfortunately, it barely trains. 
  When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly). </p>
  
  <p>Each step, the agent moves up down left right, and plot pixel or not.
  The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.</p>
  
  <p>Is there something I can try to make the agent learn visual input better?</p>
  
  <p>The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well. </p>
  
  <p><a href="https://i.sstatic.net/Zyab3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Zyab3.png" alt="agent&#39;s environment"></a><a href="https://i.sstatic.net/77SEa.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/77SEa.png" alt="Training curves"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-16T12:40:53.237Z
Comments.Text: Center image on the agent and fill the area outside of original image with black
Comments.UserDisplayName: mirror2image
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-16T13:18:58.467Z
Comments.Text: @mirror2image interesting! I'll give that a try. What would be a reason for centering making the a better representation?
Comments.UserDisplayName: SumakuTension
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-16T14:40:34.227Z
Comments.Text: Agent movement will have much bigger effect on environment and action becoming more pronounced - it more easy to detect movement of whole image then movement of small part of it.
Comments.UserDisplayName: mirror2image
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-01-16T14:57:19.587Z
Comments.Text: @mirror2image that makes sense ya. Will give it a go!
Comments.UserDisplayName: SumakuTension
Tags: <reinforcement-learning><convolutional-neural-networks>
Title: Reinforcement learning CNN input weakness
UserDisplayName: SumakuTension
ThreadText: # Reinforcement learning CNN input weakness
  
  (Asked by: username_0 on 2020-01-16)
  
  I'm trying to train a network to navigate a 48x48 2D grid, and switch pixels from on to off or off to on. The agent receives a small reward if correct, and small punishment if incorrect pixel plotted.
  
  I thought, like the Deepmind "Playing Atari with Deep Reinforcement Learning" Paper, I could just use only the pixel input, fed through 2 convolutional layers, to solve this task. The output of this is fed into 512 fully connected layer.
  
  Unfortunately, it barely trains.
  When instead using additional vectors as input containing information about nearby pixels' state around the agent, the agent learns the task quite well (yet often orients the wrong awkwardly).
  
  Each step, the agent moves up down left right, and plot pixel or not.
  The agent is visualized in the environemtn as a red square with white center dot. (also tried single red pixel). On-pixels within the red square are colored purple.
  
  Is there something I can try to make the agent learn visual input better?
  
  The orange line is the training with only visual observations, the grey one contained vector observations about the immediate neighboring pixel state as well.
  
  [![agent's environment](https://i.sstatic.net/Zyab3.png)](https://i.sstatic.net/Zyab3.png)[![Training curves](https://i.sstatic.net/77SEa.png)](https://i.sstatic.net/77SEa.png)
  
  **Tags:** reinforcement-learning, convolutional-neural-networks
  > **Comments:**
  > **username_1**: Center image on the agent and fill the area outside of original image with black  
  > **username_0**: @mirror2image interesting! I'll give that a try. What would be a reason for centering making the a better representation?  
  > **username_1**: Agent movement will have much bigger effect on environment and action becoming more pronounced - it more easy to detect movement of whole image then movement of small part of it.  
  > **username_0**: @mirror2image that makes sense ya. Will give it a go!  
  
  ---
  
  ## 0 answers
  
  