Answers.Body: <p>Yes, it is recommended to start with pre-trained model if you don't have high-end hardware. You can use a pre-trained model for fine-tuning (their trained weight as your initial weight) or use it as feature extractor (you remove few last layers, and then train it).</p>
  <p><strong>Why we need a pre-trained network?</strong></p>
  <ul>
  <li><p>Because training a good deep model takes a lot of time and needs a lot of hardware. Some good models, like DenseNet, ResNet, even VGG16 need days of training. You can read from <a href="https://arxiv.org/pdf/1409.1556.pdf" rel="nofollow noreferrer">VGG original paper</a>:</p>
  <blockquote>
  <p>On a system equipped with four NVIDIA Titan Black GPUs, training a single net took 2–3 weeks depending on the architecture.</p>
  </blockquote>
  </li>
  <li><p>Sometimes our case is similar to the dataset of pre-trained models out there. For example, if we need to classify some images of flower type, even our case has different class, we can use some first Convolutional layers of pre-trained model and use it as a feature extractor.</p>
  </li>
  </ul>
  <p>You can read more about transfer learning from <a href="https://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf" rel="nofollow noreferrer">this paper</a> or <a href="http://cs231n.github.io/transfer-learning/" rel="nofollow noreferrer">this page</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-19T09:23:53.440Z
Answers.Text: Extremely useful thank you. So I guess part of the challenge is knowing what models are available and would suit your domain?
Answers.UserDisplayName: benbyford
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-19T09:28:23.180Z
Answers.Text: that's true, you can't just pick model randomly and then use it in your case. There is a simple rule to choose the right pre-trained model, if I remember correctly it's available on the page I mentioned
Answers.UserDisplayName: malioboro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-18T22:57:22.557Z
Answers.LastActivityDate: 2019-04-18T22:57:22.557Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: malioboro
Body: <p>I quite often find projects using pre-trained model and using them as a starting point for their new model that learns something novel from thier dataset or on-live learning process - e.g. using a webcam or live audio.</p>
  
  <p>Is this quite usual and recommended to speed up training a model? For example using a model trained on ImageNet as a first layer to your model that will categorise faces specifically.</p>
  
Tags: <training><models>
Title: Is it mostly the case to train with available models
UserDisplayName: benbyford
ThreadText: # Is it mostly the case to train with available models
  
  (Asked by: username_0 on 2019-04-18)
  
  I quite often find projects using pre-trained model and using them as a starting point for their new model that learns something novel from thier dataset or on-live learning process - e.g. using a webcam or live audio.
  
  Is this quite usual and recommended to speed up training a model? For example using a model trained on ImageNet as a first layer to your model that will categorise faces specifically.
  
  **Tags:** training, models
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-18. Score: 1 | Accepted answer
  
  Yes, it is recommended to start with pre-trained model if you don't have high-end hardware. You can use a pre-trained model for fine-tuning (their trained weight as your initial weight) or use it as feature extractor (you remove few last layers, and then train it).
  
  **Why we need a pre-trained network?**
  
  * Because training a good deep model takes a lot of time and needs a lot of hardware. Some good models, like DenseNet, ResNet, even VGG16 need days of training. You can read from [VGG original paper](https://arxiv.org/pdf/1409.1556.pdf):
  
    > On a system equipped with four NVIDIA Titan Black GPUs, training a single net took 2–3 weeks depending on the architecture.
  * Sometimes our case is similar to the dataset of pre-trained models out there. For example, if we need to classify some images of flower type, even our case has different class, we can use some first Convolutional layers of pre-trained model and use it as a feature extractor.
  
  You can read more about transfer learning from [this paper](https://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-neural-networks.pdf) or [this page](http://cs231n.github.io/transfer-learning/).
  
  > **Comments:**
  > **username_0**: Extremely useful thank you. So I guess part of the challenge is knowing what models are available and would suit your domain?  
  > **username_1**: that's true, you can't just pick model randomly and then use it in your case. There is a simple rule to choose the right pre-trained model, if I remember correctly it's available on the page I mentioned  
  
  ---
  
  