Body: <p>I am using a caffe model of pre-trained <a href="https://arxiv.org/abs/1409.4842" rel="nofollow noreferrer">GoogleNet</a> trained on ImageNet from <a href="https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet" rel="nofollow noreferrer">here</a> for image retrieval task (place recognition, more specifically).</p>
  
  <p><a href="https://i.sstatic.net/5wkzB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/5wkzB.png" alt="l"></a></p>
  
  <p>I would like to know the layer with best performance in feature extraction. Its official paper suggests that:</p>
  
  <blockquote>
    <p><strong>The strong performance of shallower networks on this task suggests
    that the features produced by the <em>layers in the middle of the network</em>
    should be very discriminative.</strong></p>
  </blockquote>
  
  <p>There is also a project <a href="https://github.com/google/deepdream/blob/master/dream.ipynb" rel="nofollow noreferrer">deepdream</a> suggests that:</p>
  
  <blockquote>
    <p><strong>The complexity of the details generated depends on which layer's
    activations we try to maximize. Higher layers produce complex
    features, while lower ones enhance edges and textures, giving the
    image an impressionist feeling.</strong></p>
  </blockquote>
  
  <p>Searching the web, I found a <a href="https://github.com/colingogo/caffe-pretrained-feature-extraction" rel="nofollow noreferrer">github page</a> suggesting <code>pool5/7x7_s1</code>layer as feature extractor without specific convincing reasons.</p>
  
  <p>What I am doing now is quite cumbersome in which I extract features from each individual layer, apply scipy euclidean distance measurement to find a query in the reference database and  the judgment is based on <code>precision-recall</code> curve and my <code>top 3</code> results are as follows for one dataset:</p>
  
  <ol>
  <li><code>inception_3a/3x3</code></li>
  <li><code>inception_4a/5x5</code></li>
  <li><code>inception_4b/output</code></li>
  </ol>
  
  <p>Considering large number of convolutional layers in GoogleNet, my approach is undoubtedly quite inefficient and can be changed to another dataset!</p>
  
  <p>Can anyone suggest an efficient way to figure out the layers with the best performance as feature extractors in GoogleNet?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-14T21:19:46.020Z
Comments.Text: So, given a trained model, are you trying to understanding which of the layers of the pre-trained network extract the most "relevant" features?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-14T21:26:05.337Z
Comments.Text: That’s exactly what I’m looking for in googlenet or if there’s any rule of thumb for all networks
Comments.UserDisplayName: doplano
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-14T21:27:03.780Z
Comments.Text: Maybe you're looking for something like "layer-wise relevance propagation", which is an explainable AI technique.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-03-24T13:27:33.173Z
Comments.Text: not LRP specifically! but more of using specific layers of GoogleNet as holistic feature extraction for problems other than image classification or object detection! What I work is, for example, place recognition problem in which I investigate layers of GoogleNet to figure out ones is more efficient finding more query images in reference database!
Comments.UserDisplayName: doplano
Tags: <deep-learning><convolutional-neural-networks><computer-vision><image-processing><caffe>
Title: Is there an efficient way of determining the layers with the best performance as feature extractors in GoogleNet?
UserDisplayName: doplano
ThreadText: # Is there an efficient way of determining the layers with the best performance as feature extractors in GoogleNet?
  
  (Asked by: username_0 on 2020-03-13)
  
  I am using a caffe model of pre-trained [GoogleNet](https://arxiv.org/abs/1409.4842) trained on ImageNet from [here](https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet) for image retrieval task (place recognition, more specifically).
  
  [![l](https://i.sstatic.net/5wkzB.png)](https://i.sstatic.net/5wkzB.png)
  
  I would like to know the layer with best performance in feature extraction. Its official paper suggests that:
  
  > **The strong performance of shallower networks on this task suggests
  > that the features produced by the *layers in the middle of the network*
  > should be very discriminative.**
  
  There is also a project [deepdream](https://github.com/google/deepdream/blob/master/dream.ipynb) suggests that:
  
  > **The complexity of the details generated depends on which layer's
  > activations we try to maximize. Higher layers produce complex
  > features, while lower ones enhance edges and textures, giving the
  > image an impressionist feeling.**
  
  Searching the web, I found a [github page](https://github.com/colingogo/caffe-pretrained-feature-extraction) suggesting `pool5/7x7_s1`layer as feature extractor without specific convincing reasons.
  
  What I am doing now is quite cumbersome in which I extract features from each individual layer, apply scipy euclidean distance measurement to find a query in the reference database and the judgment is based on `precision-recall` curve and my `top 3` results are as follows for one dataset:
  
  1. `inception_3a/3x3`
  2. `inception_4a/5x5`
  3. `inception_4b/output`
  
  Considering large number of convolutional layers in GoogleNet, my approach is undoubtedly quite inefficient and can be changed to another dataset!
  
  Can anyone suggest an efficient way to figure out the layers with the best performance as feature extractors in GoogleNet?
  
  **Tags:** deep-learning, convolutional-neural-networks, computer-vision, image-processing, caffe
  
  > **Comments:**
  > **username_1**: So, given a trained model, are you trying to understanding which of the layers of the pre-trained network extract the most "relevant" features?  
  > **username_0**: That’s exactly what I’m looking for in googlenet or if there’s any rule of thumb for all networks  
  > **username_1**: Maybe you're looking for something like "layer-wise relevance propagation", which is an explainable AI technique.  
  > **username_0**: not LRP specifically! but more of using specific layers of GoogleNet as holistic feature extraction for problems other than image classification or object detection! What I work is, for example, place recognition problem in which I investigate layers of GoogleNet to figure out ones is more efficient finding more query images in reference database!  
  
  ---
  
  ## 0 answers
  
  