Answers.Body: <p>We can't say for sure which approach would work best in the general case. If you have domain knowledge, you can make a better guess. You'll basically want to answer the question: which information is important for learning an optimal policy?</p>
  <blockquote>
  <p>In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.</p>
  </blockquote>
  <p>Generally, if you have an environment like this, I would (without any other information) guess that each of the 5 colors have some meaning that may be relevant for your agent. That's just what I would guess though. In theory, it might be possible that white means one thing (e.g. &quot;empty&quot;), and every other colour means the same other thing (e.g. &quot;not empty&quot;). If you had domain knowledge like that, and knew that it is only important whether or not any given pixel is white, you could of course binarise your input.</p>
  <p>But in general, if the colours might be important, I'd recommend including them. If you really only have a few distinct colours like that though, I would not recommend encoding them in some format like RGB where values can range from 0 to 1 or 0 to 255. I would recommend having 4 (or 5?) binary channels in your input:</p>
  <ol>
  <li>Binary channel containing 1s for pixels that are black, and 0s for all other pixels.</li>
  <li>Binary channel containing 1s for pixels that are white, and 0s for all other pixels.</li>
  <li>Binary channel containing 1s for pixels that are blue, and 0s for all other pixels.</li>
  <li>...</li>
  <li>etc.</li>
  </ol>
  <p>The reason for this is that deep neural networks often tend to have an easier time learning with binary inputs, and here you can completely binarise your inputs without requiring an excessively high number of channels. If you had hundreds or thousands of different possible colours, this would probably no longer be a good idea.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-19T18:28:26.483Z
Answers.LastActivityDate: 2021-02-03T17:37:48.660Z
Answers.LastEditDate: 2021-02-03T17:37:48.660Z
Answers.UserDisplayName: Dennis Soemers
Body: <p>I'm doing reinforcement learning, and I have a visual observation that I will use to build an input state for my agent. In the DeepMind's Atari paper, they greyscale the input image before they input it into the CNN to reduce the input space's size, which makes sense to me.</p>
  <p>In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.</p>
  <p>Any thoughts on what could be better? Greyscaling into 2 shades of grey and black and white also maintains the information, but feels somehow less direct, since my environment's visual space is categorical, which makes more sense in a categorical encoding.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-21T05:36:55.853Z
Comments.Text: Greyscaling can only make sense, when the meaning of blue, red and green is somewhere between white and black. Otherwise, you'd force the network to decode it first. And even when the meaning is like above, providing one bit for each possible state sounds better. That's how networks for chess work: one bit per piece kind and field.
Comments.UserDisplayName: maaartinus
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-03T17:40:12.457Z
Comments.Text: I've edited this post in order to clarify it further and to improve the title. Please, make sure I didn't change the meaning of it.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><convolutional-neural-networks><dqn><deep-rl><data-preprocessing>
Title: Should I grey-scale the coloured frames/channels to build the approximation of the state?
UserDisplayName: SumakuTension
ThreadText: # Should I grey-scale the coloured frames/channels to build the approximation of the state?
  
  (Asked by: username_0 on 2020-02-19)
  
  I'm doing reinforcement learning, and I have a visual observation that I will use to build an input state for my agent. In the DeepMind's Atari paper, they greyscale the input image before they input it into the CNN to reduce the input space's size, which makes sense to me.
  
  In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.
  
  Any thoughts on what could be better? Greyscaling into 2 shades of grey and black and white also maintains the information, but feels somehow less direct, since my environment's visual space is categorical, which makes more sense in a categorical encoding.
  
  **Tags:** reinforcement-learning, convolutional-neural-networks, dqn, deep-rl, data-preprocessing
  > **Comments:**
  > **username_1**: Greyscaling can only make sense, when the meaning of blue, red and green is somewhere between white and black. Otherwise, you'd force the network to decode it first. And even when the meaning is like above, providing one bit for each possible state sounds better. That's how networks for chess work: one bit per piece kind and field.  
  > **username_2**: I've edited this post in order to clarify it further and to improve the title. Please, make sure I didn't change the meaning of it.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2020-02-19. Score: 4 | Accepted answer
  
  We can't say for sure which approach would work best in the general case. If you have domain knowledge, you can make a better guess. You'll basically want to answer the question: which information is important for learning an optimal policy?
  
  > In my environment, I have, for each pixel, 5 possible channels, which are represented in black, white, blue, red, and green. This also makes intuitive sense to me since it's like a bit-encoding.
  
  Generally, if you have an environment like this, I would (without any other information) guess that each of the 5 colors have some meaning that may be relevant for your agent. That's just what I would guess though. In theory, it might be possible that white means one thing (e.g. "empty"), and every other colour means the same other thing (e.g. "not empty"). If you had domain knowledge like that, and knew that it is only important whether or not any given pixel is white, you could of course binarise your input.
  
  But in general, if the colours might be important, I'd recommend including them. If you really only have a few distinct colours like that though, I would not recommend encoding them in some format like RGB where values can range from 0 to 1 or 0 to 255. I would recommend having 4 (or 5?) binary channels in your input:
  
  1. Binary channel containing 1s for pixels that are black, and 0s for all other pixels.
  2. Binary channel containing 1s for pixels that are white, and 0s for all other pixels.
  3. Binary channel containing 1s for pixels that are blue, and 0s for all other pixels.
  4. ...
  5. etc.
  
  The reason for this is that deep neural networks often tend to have an easier time learning with binary inputs, and here you can completely binarise your inputs without requiring an excessively high number of channels. If you had hundreds or thousands of different possible colours, this would probably no longer be a good idea.
  
  
  ---
  
  