Answers.Body: <p>The Faster R-CNN from the TensorFlow Hub is an object detection model, not just a classification model. That explains why it has 8 outputs, including softmax probabilities and bounding boxes. If you only want the probabilities, I suggest the following code :</p>
  <pre><code>n_classes = 2
  
  model_handle = &quot;https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1&quot;
  k_layer = hub.KerasLayer(model_handle, trainable=True)
  
  inputs = tf.keras.layers.Input(shape=(640, 640, 3), dtype=tf.uint8)
  outputs_k_layer = k_layer(inputs)
  
  # To enhance your model, you can add as many layers as you want
  detection_multiclass_scores = tf.keras.layers.Dense(128, activation='relu')(outputs_k_layer['detection_multiclass_scores'])
  detection_multiclass_scores = tf.keras.layers.Dropout(0.2)(detection_multiclass_scores)
  detection_multiclass_scores = tf.keras.layers.Dense(n_classes)(detection_multiclass_scores)
  
  classifier = tf.keras.Model(inputs=inputs, outputs={'detection_multiclass_scores' : detection_multiclass_scores})
  </code></pre>
  <p>You can also get the bounding boxes. The bounding boxes are regressions independent of the nature of the object we want to detect, so they don't require transfer learning, unlike the scores related to each class :</p>
  <pre><code>outputs = {
      'detection_multiclass_scores' : detection_multiclass_scores,
      'detection_boxes': outputs_k_layer['detection_boxes'],
  }
  object_detector = tf.keras.Model(inputs=inputs, outputs=outputs)
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-17T16:59:36.710Z
Answers.Text: Many thanks @abkg. How do you customize the Faster R-CNN from the TensorFlow Hub then? Your explanation helps me extract certain outputs but I would like to add new layers for further training and also want to only classify say 5 object types as opposed to the hundreds of different objects that the model can detect.
Answers.UserDisplayName: Doug
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-18T14:12:33.230Z
Answers.Text: To enhance your model, add layers before the final dense one. Initially, freeze pre-trained layers, training only the new layers for your classes, to avoid major weight shifts. After initial training, optionally unfreeze some or all pre-trained layers for fine-tuning. Ensure your dataset contains only your 5 object classes before training
Answers.UserDisplayName: abkg
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-18T15:29:37.163Z
Answers.Text: Thanks again @abkg Would it be possible to expand your example to show how I could add a layer before the final dense one?
Answers.UserDisplayName: Doug
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-19T10:37:51.083Z
Answers.Text: I added an additional dense layer and a dropout layer for the sake of example.
Answers.UserDisplayName: abkg
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-19T15:57:39.730Z
Answers.Text: Thank you very much for all your help and guidance @abkg. I accepted your post as the answer. I would be grateful if you could answer one final question. In your example you add the layers before the detection_multiclass_scores output but would that mean it adds layers for other outputs as well? If not, one output has more layers than others?
Answers.UserDisplayName: Doug
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-19T21:34:15.687Z
Answers.Text: In my example, other outputs of the model are not affected by the new layers, but you could make it so. It's up to you and your objective. To do that, just add new layers to the `detection_boxes` as I did with `detection_multiclass_scores`.
Answers.UserDisplayName: abkg
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-16T22:26:22.297Z
Answers.LastActivityDate: 2023-10-19T10:36:23.317Z
Answers.LastEditDate: 2023-10-19T10:36:23.317Z
Answers.UserDisplayName: abkg
Body: <p>I am new to AI/ML and wanted to seek guidance as I am totally lost. I will simplify my issue as follows:</p>
  <p>Let's say I would like to detect apples and oranges in images.
  I would like to leverage a pre-trained Faster RCNN model for efficiency.
  My understanding is that I need to remove the last two layers of such a model and add my own custom layers to detect what I want to detect. This gives me the option to tailor the model to my needs.
  However, these models typically have 8 outputs or so with things like detection_boxes, detection_classes, etc.</p>
  <p>My question is then how do I customize the model to focus on two object classes and feed additional datasets to further train it? In other words, use the learnings from pre-training but customize the model and further train based on my data.</p>
  <p>I can load the model with the following code:</p>
  <pre><code>input_shape = (640, 640, 3)
  
  model_handle = &quot;https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1&quot;
  k_layer = hub.KerasLayer(model_handle, input_shape=(None, None, 3))
  
  inputs = tf.keras.layers.Input(shape=(None, None, 3), dtype=tf.uint8)
  outputs_k_layer = k_layer(inputs)
  
  classifier = tf.keras.Model(inputs, outputs_k_layer)
  
  print(classifier.summary()) 
  </code></pre>
  <p>However, I cannot add a new layer as new layers require one input and output as opposed to the 8 outputs...</p>
  <p>For example something like below throws an error</p>
  <pre><code>x = tf.keras.layers.Dense(128, activation=&quot;relu&quot;)(outputs_k_layer)
  </code></pre>
  <pre><code>Error: Layer &quot;dense_2&quot; expects 1 input(s), but it received 8 input tensors. 
  </code></pre>
  <p>I think I am confusing concepts and totally lost my way. I would be grateful if you could point me towards the right direction.</p>
  <p>Best,</p>
  <p>Doug</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-16T23:39:46.263Z
Comments.Text: crosspost: https://stackoverflow.com/questions/77303330/customizing-keraslayer-for-transfer-learning-wrt-tensorflow-pretrained-models
Comments.UserDisplayName: Christoph Rackwitz
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-17T16:55:28.137Z
Comments.Text: I deleted the other post, thanks for pointing it out
Comments.UserDisplayName: Doug
Tags: <convolutional-neural-networks><tensorflow><keras><object-detection><models>
Title: Transfer learning using pretrained tensorflow object detection model
UserDisplayName: Doug
ThreadText: # Transfer learning using pretrained tensorflow object detection model
  
  (Asked by: username_0 on 2023-10-16)
  
  I am new to AI/ML and wanted to seek guidance as I am totally lost. I will simplify my issue as follows:
  
  Let's say I would like to detect apples and oranges in images.
  I would like to leverage a pre-trained Faster RCNN model for efficiency.
  My understanding is that I need to remove the last two layers of such a model and add my own custom layers to detect what I want to detect. This gives me the option to tailor the model to my needs.
  However, these models typically have 8 outputs or so with things like detection\_boxes, detection\_classes, etc.
  
  My question is then how do I customize the model to focus on two object classes and feed additional datasets to further train it? In other words, use the learnings from pre-training but customize the model and further train based on my data.
  
  I can load the model with the following code:
  
  ```
  input_shape = (640, 640, 3)
  
  model_handle = "https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1"
  k_layer = hub.KerasLayer(model_handle, input_shape=(None, None, 3))
  
  inputs = tf.keras.layers.Input(shape=(None, None, 3), dtype=tf.uint8)
  outputs_k_layer = k_layer(inputs)
  
  classifier = tf.keras.Model(inputs, outputs_k_layer)
  
  print(classifier.summary()) 
  
  ```
  
  However, I cannot add a new layer as new layers require one input and output as opposed to the 8 outputs...
  
  For example something like below throws an error
  
  ```
  x = tf.keras.layers.Dense(128, activation="relu")(outputs_k_layer)
  
  ```
  
  ```
  Error: Layer "dense_2" expects 1 input(s), but it received 8 input tensors. 
  
  ```
  
  I think I am confusing concepts and totally lost my way. I would be grateful if you could point me towards the right direction.
  
  Best,
  
  username_0
  
  **Tags:** convolutional-neural-networks, tensorflow, keras, object-detection, models
  
  > **Comments:**
  > **username_1**: crosspost: https://stackoverflow.com/questions/77303330/customizing-keraslayer-for-transfer-learning-wrt-tensorflow-pretrained-models  
  > **username_0**: I deleted the other post, thanks for pointing it out  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-10-16. Score: 1 | Accepted answer
  
  The Faster R-CNN from the TensorFlow Hub is an object detection model, not just a classification model. That explains why it has 8 outputs, including softmax probabilities and bounding boxes. If you only want the probabilities, I suggest the following code :
  
  ```
  n_classes = 2
  
  model_handle = "https://tfhub.dev/tensorflow/faster_rcnn/inception_resnet_v2_640x640/1"
  k_layer = hub.KerasLayer(model_handle, trainable=True)
  
  inputs = tf.keras.layers.Input(shape=(640, 640, 3), dtype=tf.uint8)
  outputs_k_layer = k_layer(inputs)
  
  # To enhance your model, you can add as many layers as you want
  detection_multiclass_scores = tf.keras.layers.Dense(128, activation='relu')(outputs_k_layer['detection_multiclass_scores'])
  detection_multiclass_scores = tf.keras.layers.Dropout(0.2)(detection_multiclass_scores)
  detection_multiclass_scores = tf.keras.layers.Dense(n_classes)(detection_multiclass_scores)
  
  classifier = tf.keras.Model(inputs=inputs, outputs={'detection_multiclass_scores' : detection_multiclass_scores})
  
  ```
  
  You can also get the bounding boxes. The bounding boxes are regressions independent of the nature of the object we want to detect, so they don't require transfer learning, unlike the scores related to each class :
  
  ```
  outputs = {
      'detection_multiclass_scores' : detection_multiclass_scores,
      'detection_boxes': outputs_k_layer['detection_boxes'],
  }
  object_detector = tf.keras.Model(inputs=inputs, outputs=outputs)
  
  ```
  
  > **Comments:**
  > **username_0**: Many thanks @username_2. How do you customize the Faster R-CNN from the TensorFlow Hub then? Your explanation helps me extract certain outputs but I would like to add new layers for further training and also want to only classify say 5 object types as opposed to the hundreds of different objects that the model can detect.  
  > **username_2**: To enhance your model, add layers before the final dense one. Initially, freeze pre-trained layers, training only the new layers for your classes, to avoid major weight shifts. After initial training, optionally unfreeze some or all pre-trained layers for fine-tuning. Ensure your dataset contains only your 5 object classes before training  
  > **username_0**: Thanks again @username_2 Would it be possible to expand your example to show how I could add a layer before the final dense one?  
  > **username_2**: I added an additional dense layer and a dropout layer for the sake of example.  
  > **username_0**: Thank you very much for all your help and guidance @username_2. I accepted your post as the answer. I would be grateful if you could answer one final question. In your example you add the layers before the detection_multiclass_scores output but would that mean it adds layers for other outputs as well? If not, one output has more layers than others?  
  > **username_2**: In my example, other outputs of the model are not affected by the new layers, but you could make it so. It's up to you and your objective. To do that, just add new layers to the `detection_boxes` as I did with `detection_multiclass_scores`.  
  
  ---
  
  