Answers.Body: <p>I can judge from the excerpt only, but I don't see any mention of a convolutional layer there. This doesn't mean that convolution isn't suitable for this task (in fact, it's the best method for image classification so far), it's just seems the instructor proposes this particular network architecture, without convolutions.</p>
  
  <p>Since this architecture is more illustrative and by no means a standard, I can't be sure I fully understand it. But it seems like the first layer outputs a vector of size <code>4</code>, which contains the counts of four <em>selected</em> colors. The ReLu operation here is a fancy way of filtering extra colors, based on channel value (note that they are close to either <code>0</code> or <code>255</code> in all four colors). If you select different four colors, this filtering will have to be done differently too, so it's basically hardwired. The second layer is more straightforward.</p>
  
  <p>In general, you shouldn't consider this a "true" deep network, but merely a puzzle to get familiar with ReLu operation.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-08T08:12:04.967Z
Answers.LastActivityDate: 2017-10-08T08:12:04.967Z
Answers.UserDisplayName: Maxim
Body: <p>Below is an excerpt in an instructor's manual on ML that is explaining deep neural networks, using cat recognition (what else!) from images as example. On how DL performs this feat, the excerpt said that,</p>
  
  <blockquote>
    <p>Assume that the first layer returns the number of pixels that are
    brown/black/blue/red, and the second layer finds the most common
    color, and the third layer returns “cat” if previous layer had
    supplied “brown”. [..] Mathematically, this model would be, for the
    first layer, [ sum(r = 255, g=255, b=255), ..., ..., sum(r=255, g=0,
    b=0)] -- this is just a set of appropriately positioned relu functions
    (okay, for r=234, we’d need two relu functions, so two layers, but you
    get the idea). The second layer would be a softmax layer. The third
    layer is simply an identity!</p>
  </blockquote>
  
  <p>Now I worked with deep nets, but I am not sure how I can structure a DL to do this. ReLu is simply a max(0,x), so how would I filter out pixel vals for example 128,128,128 and sum them up? Wouldn't the convolution layer play a role here too? What would the layout of a simple deep net be that does what is described above?</p>
  
  <p>Thanks,</p>
  
Tags: <neural-networks><deep-learning><convolutional-neural-networks>
Title: ReLu, Sum and Convolution Layers to Count Pixels of Certain Color
UserDisplayName: BBSysDyn
ThreadText: # ReLu, Sum and Convolution Layers to Count Pixels of Certain Color
  
  (Asked by: username_0 on 2017-09-19)
  
  Below is an excerpt in an instructor's manual on ML that is explaining deep neural networks, using cat recognition (what else!) from images as example. On how DL performs this feat, the excerpt said that,
  
  > Assume that the first layer returns the number of pixels that are
  > brown/black/blue/red, and the second layer finds the most common
  > color, and the third layer returns “cat” if previous layer had
  > supplied “brown”. [..] Mathematically, this model would be, for the
  > first layer, [ sum(r = 255, g=255, b=255), ..., ..., sum(r=255, g=0,
  > b=0)] -- this is just a set of appropriately positioned relu functions
  > (okay, for r=234, we’d need two relu functions, so two layers, but you
  > get the idea). The second layer would be a softmax layer. The third
  > layer is simply an identity!
  
  Now I worked with deep nets, but I am not sure how I can structure a DL to do this. ReLu is simply a max(0,x), so how would I filter out pixel vals for example 128,128,128 and sum them up? Wouldn't the convolution layer play a role here too? What would the layout of a simple deep net be that does what is described above?
  
  Thanks,
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-10-08. Score: 0 | Accepted answer
  
  I can judge from the excerpt only, but I don't see any mention of a convolutional layer there. This doesn't mean that convolution isn't suitable for this task (in fact, it's the best method for image classification so far), it's just seems the instructor proposes this particular network architecture, without convolutions.
  
  Since this architecture is more illustrative and by no means a standard, I can't be sure I fully understand it. But it seems like the first layer outputs a vector of size `4`, which contains the counts of four *selected* colors. The ReLu operation here is a fancy way of filtering extra colors, based on channel value (note that they are close to either `0` or `255` in all four colors). If you select different four colors, this filtering will have to be done differently too, so it's basically hardwired. The second layer is more straightforward.
  
  In general, you shouldn't consider this a "true" deep network, but merely a puzzle to get familiar with ReLu operation.
  
  
  ---
  
  