Answers.Body: <p>Besides the answer from @GKozinski, you can also fine tune all the layers of the pretrained model.</p>
  <p>Yes, it is incredibly commonplace to do transfer learning using pretrained models from different domains.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-29T03:06:18.730Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-29T14:25:02.843Z
Answers.Text: Could you provide more reading articles on fine tuning all the layers? In object detection API of TensorFlow, I do not really see any layers.
Answers.UserDisplayName: SunnyBoiz
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-29T01:01:05.423Z
Answers.LastActivityDate: 2022-08-29T01:01:05.423Z
Answers.UserDisplayName: Joseph Turian
Answers.Body: <p>Well, I think you forget about &quot;fine tuning&quot; stage here. What they mean in these tutorials is that you take such model that was pretrained on large dataset and you usually freeze from training all layers except the last one or few last ones and you train these last layer/layers on your smaller specific dataset. This is called &quot;transfer learning&quot;.</p>
  <p>So in theory model should learn more general features during that first trainin and use them without changes in the second training (fine tuning), when it learns features more specific to your problem. Only then you can use it for detection objects from your dataset. If the model has never seen a labeled part from your dataset during its training it will never come up what this is by itself that's why you need this fine tuning process. But fortunately it requires far less data and thanks to that is much faster too.</p>
  <p>Usually you let fine tune only last layer. But if you feel that your problem is noticeably different from the images the model been trained on then you can try tuning two or three last layers.</p>
  <p>It's not alaways easy to find a model pretrained on dataset from every possible domain so usually you just start with a model pretrained on datasets containing images of very general classes of objects such as ImageNet or COCO.</p>
  <p>If you work on a model from scratch it would probably require a lot of data. It's easy to overfit deep learning models with just few hundreds samples.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-26T05:39:47.023Z
Answers.Text: Thanks for your reply. I tried to find information regarding freezing the last layer you mentioned but TensorFlow objection detection API does not seem to have these layers in the pipeline.config (I am using Tensorflow 2 Model Zoo). Do you have a website which clearly show this? I see there are example in Keras but I am not using Keras.
Answers.UserDisplayName: SunnyBoiz
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-26T08:19:56.203Z
Answers.Text: You don't freeze the last one but all previous ones except last one. I haven't used this object detection API for a long time but I am pretty sure these weights are already freezed by default. I followed these tutorials and it was working for me all fine: https://youtu.be/yqkISICHH-U and https://youtube.com/playlist?list=PLQVvvaa0QuDcNK5GeCQnxYnSSaar2tpku
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-26T08:23:10.963Z
Answers.Text: @SunnyBoiz  Also if your goal is to make a good object detection model and you dont care what models you use I would recommend Ultralytic's implementation of YOLOv5 instead of Tensorflow object detection API. Much smoother and easier to setup and better results.
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-26T05:12:51.113Z
Answers.LastActivityDate: 2022-08-26T05:12:51.113Z
Answers.UserDisplayName: GKozinski
Body: <p>I have been trying out various tutorials on object detection machine learning. All the tutorials so far have been to use a pre-trained model for practical reasons when detecting objects that the pre-trained model learnt (e.g cats &amp; dogs). However, will this pre-trained model work if I input a few hundred images of a particular car engine part and predict this class, which the pre-trained model did not train on? Is it recommended to make a model from scratch in this case?</p>
  <p>I am further confused by this in TensorFlow documentation (Images -&gt; Transfer learning and fine-tuning), the summary states:</p>
  <blockquote>
  <p>Using a pre-trained model for feature extraction: When working with a small dataset, it is a common practice to take advantage of features learned by a model trained on a larger dataset in the <strong>same domain</strong></p>
  </blockquote>
  <p>By that meaning, if I need to predict a particular car engine part then this statement seems to suggest I create a model from scratch?</p>
  <p>TLDR: Will pre-trained model be able to work on image dataset that it has never learn before or better to work on a model from scratch?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-26T03:59:43.010Z
Comments.Text: you might want to look at: https://deepai.org/machine-learning-model/text2img - this is what is produced from typing "car engine"  https://i.sstatic.net/mN8K2.jpg
Comments.UserDisplayName: Rob
Tags: <convolutional-neural-networks><object-detection><image-segmentation>
Title: Using pre-trained models on image dataset that is totally different for object detection?
UserDisplayName: SunnyBoiz
ThreadText: # Using pre-trained models on image dataset that is totally different for object detection?
  
  (Asked by: username_0 on 2022-08-26)
  
  I have been trying out various tutorials on object detection machine learning. All the tutorials so far have been to use a pre-trained model for practical reasons when detecting objects that the pre-trained model learnt (e.g cats & dogs). However, will this pre-trained model work if I input a few hundred images of a particular car engine part and predict this class, which the pre-trained model did not train on? Is it recommended to make a model from scratch in this case?
  
  I am further confused by this in TensorFlow documentation (Images -> Transfer learning and fine-tuning), the summary states:
  
  > Using a pre-trained model for feature extraction: When working with a small dataset, it is a common practice to take advantage of features learned by a model trained on a larger dataset in the **same domain**
  
  By that meaning, if I need to predict a particular car engine part then this statement seems to suggest I create a model from scratch?
  
  TLDR: Will pre-trained model be able to work on image dataset that it has never learn before or better to work on a model from scratch?
  
  **Tags:** convolutional-neural-networks, object-detection, image-segmentation
  
  > **Comments:**
  > **username_1**: you might want to look at: https://deepai.org/machine-learning-model/text2img - this is what is produced from typing "car engine"  https://i.sstatic.net/mN8K2.jpg  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2022-08-26. Score: 2 | Accepted answer
  
  Well, I think you forget about "fine tuning" stage here. What they mean in these tutorials is that you take such model that was pretrained on large dataset and you usually freeze from training all layers except the last one or few last ones and you train these last layer/layers on your smaller specific dataset. This is called "transfer learning".
  
  So in theory model should learn more general features during that first trainin and use them without changes in the second training (fine tuning), when it learns features more specific to your problem. Only then you can use it for detection objects from your dataset. If the model has never seen a labeled part from your dataset during its training it will never come up what this is by itself that's why you need this fine tuning process. But fortunately it requires far less data and thanks to that is much faster too.
  
  Usually you let fine tune only last layer. But if you feel that your problem is noticeably different from the images the model been trained on then you can try tuning two or three last layers.
  
  It's not alaways easy to find a model pretrained on dataset from every possible domain so usually you just start with a model pretrained on datasets containing images of very general classes of objects such as ImageNet or COCO.
  
  If you work on a model from scratch it would probably require a lot of data. It's easy to overfit deep learning models with just few hundreds samples.
  
  > **Comments:**
  > **username_0**: Thanks for your reply. I tried to find information regarding freezing the last layer you mentioned but TensorFlow objection detection API does not seem to have these layers in the pipeline.config (I am using Tensorflow 2 Model Zoo). Do you have a website which clearly show this? I see there are example in Keras but I am not using Keras.  
  > **username_4**: You don't freeze the last one but all previous ones except last one. I haven't used this object detection API for a long time but I am pretty sure these weights are already freezed by default. I followed these tutorials and it was working for me all fine: https://youtu.be/yqkISICHH-U and https://youtube.com/playlist?list=PLQVvvaa0QuDcNK5GeCQnxYnSSaar2tpku  
  > **username_4**: @username_0  Also if your goal is to make a good object detection model and you dont care what models you use I would recommend Ultralytic's implementation of YOLOv5 instead of Tensorflow object detection API. Much smoother and easier to setup and better results.  
  
  ---
  
  ### Answer by username_2 on 2022-08-29. Score: 0
  
  Besides the answer from @username_4, you can also fine tune all the layers of the pretrained model.
  
  Yes, it is incredibly commonplace to do transfer learning using pretrained models from different domains.
  
  > **Comments:**
  > **username_3**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  > **username_0**: Could you provide more reading articles on fine tuning all the layers? In object detection API of TensorFlow, I do not really see any layers.  
  
  ---
  
  