Answers.Body: <p>Pooling doesn't completely remove information about location of features within your image. If you don't want to use pooling but want to reduce the size of your neural net layers you should try a value of stride greater than 1 in your convolutional layers. e.g:</p>
  
  <p><code>keras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)
  </code></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-19T06:48:06.900Z
Answers.Text: Increasing the side is very similar to pooling. In fact you can represent pooling by the convolutional layers
Answers.UserDisplayName: Martin Thoma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-19T12:15:52.403Z
Answers.Text: @MartinThoma where in my answer did I say that stride is not similar to pooling?
Answers.UserDisplayName: Mike Vella
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-19T13:14:11.860Z
Answers.LastActivityDate: 2018-09-19T13:14:11.860Z
Answers.UserDisplayName: Mike Vella
Answers.Body: <p>1X1 convolution might improve accuracy as they reduce dimension in filter space.</p>
  
  <p>Google inception V3 architecture would be a good starting point.</p>
  
  <p><a href="https://arxiv.org/pdf/1512.00567.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1512.00567.pdf</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-19T22:46:56.383Z
Answers.LastActivityDate: 2018-09-19T22:46:56.383Z
Answers.UserDisplayName: Sreedhar Veluri
Body: <p>I'm trying to use a CNN to analyse statistical images. These images are not 'natural' images (cats, dogs, etc) but images generated by visualising a dataset. The idea is that these datasets hopefully contain patterns in them that can be used as part of a classification problem.</p>
  
  <p><a href="https://i.sstatic.net/YGYUZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/YGYUZ.png" alt="enter image description here"></a></p>
  
  <p>Most CNN examples I've seen have one of more pooling layers, and the explaination I've seen for them is to reduce the number of training elements, but also to allow for some locational independance of an element (e.g. I know this is an eye, and can appear anywhere in the image).</p>
  
  <p>In my case location <em>is</em> important and I want my CNN to be aware of that. ie. the presence of a pattern at a specific location in the image means something very specific compared to if that feature or pattern appears elsewhere.</p>
  
  <p>At the moment my network looks like this (taken from an example somewhere):</p>
  
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  conv2d_1 (Conv2D)            (None, 196, 178, 32)      896       
  _________________________________________________________________
  activation_1 (Activation)    (None, 196, 178, 32)      0         
  _________________________________________________________________
  max_pooling2d_1 (MaxPooling2 (None, 98, 89, 32)        0         
  _________________________________________________________________
  conv2d_2 (Conv2D)            (None, 96, 87, 32)        9248      
  _________________________________________________________________
  activation_2 (Activation)    (None, 96, 87, 32)        0         
  _________________________________________________________________
  max_pooling2d_2 (MaxPooling2 (None, 48, 43, 32)        0         
  _________________________________________________________________
  conv2d_3 (Conv2D)            (None, 46, 41, 64)        18496     
  _________________________________________________________________
  activation_3 (Activation)    (None, 46, 41, 64)        0         
  _________________________________________________________________
  max_pooling2d_3 (MaxPooling2 (None, 23, 20, 64)        0         
  _________________________________________________________________
  flatten_1 (Flatten)          (None, 29440)             0         
  _________________________________________________________________
  dense_1 (Dense)              (None, 32)                942112    
  _________________________________________________________________
  activation_4 (Activation)    (None, 32)                0         
  _________________________________________________________________
  dropout_1 (Dropout)          (None, 32)                0         
  _________________________________________________________________
  dense_2 (Dense)              (None, 3)                 99        
  _________________________________________________________________
  activation_5 (Activation)    (None, 3)                 0         
  =================================================================
  Total params: 970,851
  Trainable params: 970,851
  Non-trainable params: 0
  _________________________________________________________________
  </code></pre>
  
  <p>The 'images' I'm training on are 180 x 180 x 3 pixels and each channel contains a different set of raw data.</p>
  
  <p>What strategies are there to improve my CNN to deal with this? I have tried simply removing some of the pooling layers, but that greatly increased memory and training time and didn't seem to really help.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-19T06:43:07.020Z
Comments.Text: Can you tell me more about statistical images? How should I interpret the image you posted?
Comments.UserDisplayName: Martin Thoma
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-19T06:46:14.433Z
Comments.Text: How good is your current network? Which type of mistakes does it make? Have a look at [chapter 2.5](https://arxiv.org/pdf/1707.09725) of my Master's thesis for some ideas how to analyze your network.
Comments.UserDisplayName: Martin Thoma
Tags: <deep-learning><convolutional-neural-networks><keras>
Title: CNN Pooling layers unhelpful when location important?
UserDisplayName: Matt Hamilton
ThreadText: # CNN Pooling layers unhelpful when location important?
  
  (Asked by: username_0 on 2018-09-19)
  
  I'm trying to use a CNN to analyse statistical images. These images are not 'natural' images (cats, dogs, etc) but images generated by visualising a dataset. The idea is that these datasets hopefully contain patterns in them that can be used as part of a classification problem.
  
  [![enter image description here](https://i.sstatic.net/YGYUZ.png)](https://i.sstatic.net/YGYUZ.png)
  
  Most CNN examples I've seen have one of more pooling layers, and the explaination I've seen for them is to reduce the number of training elements, but also to allow for some locational independance of an element (e.g. I know this is an eye, and can appear anywhere in the image).
  
  In my case location *is* important and I want my CNN to be aware of that. ie. the presence of a pattern at a specific location in the image means something very specific compared to if that feature or pattern appears elsewhere.
  
  At the moment my network looks like this (taken from an example somewhere):
  
  ```
  _________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  conv2d_1 (Conv2D)            (None, 196, 178, 32)      896       
  _________________________________________________________________
  activation_1 (Activation)    (None, 196, 178, 32)      0         
  _________________________________________________________________
  max_pooling2d_1 (MaxPooling2 (None, 98, 89, 32)        0         
  _________________________________________________________________
  conv2d_2 (Conv2D)            (None, 96, 87, 32)        9248      
  _________________________________________________________________
  activation_2 (Activation)    (None, 96, 87, 32)        0         
  _________________________________________________________________
  max_pooling2d_2 (MaxPooling2 (None, 48, 43, 32)        0         
  _________________________________________________________________
  conv2d_3 (Conv2D)            (None, 46, 41, 64)        18496     
  _________________________________________________________________
  activation_3 (Activation)    (None, 46, 41, 64)        0         
  _________________________________________________________________
  max_pooling2d_3 (MaxPooling2 (None, 23, 20, 64)        0         
  _________________________________________________________________
  flatten_1 (Flatten)          (None, 29440)             0         
  _________________________________________________________________
  dense_1 (Dense)              (None, 32)                942112    
  _________________________________________________________________
  activation_4 (Activation)    (None, 32)                0         
  _________________________________________________________________
  dropout_1 (Dropout)          (None, 32)                0         
  _________________________________________________________________
  dense_2 (Dense)              (None, 3)                 99        
  _________________________________________________________________
  activation_5 (Activation)    (None, 3)                 0         
  =================================================================
  Total params: 970,851
  Trainable params: 970,851
  Non-trainable params: 0
  _________________________________________________________________
  
  ```
  
  The 'images' I'm training on are 180 x 180 x 3 pixels and each channel contains a different set of raw data.
  
  What strategies are there to improve my CNN to deal with this? I have tried simply removing some of the pooling layers, but that greatly increased memory and training time and didn't seem to really help.
  
  **Tags:** deep-learning, convolutional-neural-networks, keras
  
  > **Comments:**
  > **username_1**: Can you tell me more about statistical images? How should I interpret the image you posted?  
  > **username_1**: How good is your current network? Which type of mistakes does it make? Have a look at [chapter 2.5](https://arxiv.org/pdf/1707.09725) of my Master's thesis for some ideas how to analyze your network.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2018-09-19. Score: -1
  
  Pooling doesn't completely remove information about location of features within your image. If you don't want to use pooling but want to reduce the size of your neural net layers you should try a value of stride greater than 1 in your convolutional layers. e.g:
  
  `keras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)`
  
  > **Comments:**
  > **username_1**: Increasing the side is very similar to pooling. In fact you can represent pooling by the convolutional layers  
  > **username_2**: @username_1 where in my answer did I say that stride is not similar to pooling?  
  
  ---
  
  ### Answer by username_3 on 2018-09-19. Score: -1
  
  1X1 convolution might improve accuracy as they reduce dimension in filter space.
  
  Google inception V3 architecture would be a good starting point.
  
  <https://arxiv.org/pdf/1512.00567.pdf>
  
  
  ---
  
  