Answers.Body: <p>After a lot of browsing online for answers to these questions, this is what I came up with.</p>
  <blockquote>
  <ol>
  <li>What are the initial layers in this case? How exactly can I freeze them?</li>
  </ol>
  </blockquote>
  <p>The <strong>initial few layers</strong> are said to extract the most general features of any kind of image, like edges or corners of objects. So, I guess it actually would depend on the kind of <a href="https://www.analyticsvidhya.com/blog/2017/08/10-advanced-deep-learning-architectures-data-scientists/" rel="nofollow noreferrer">backbone architecture</a> you are selecting.</p>
  <p>How to freeze the layers depends on the framework we use.</p>
  <p>(I have selected <a href="http://pytorch.org" rel="nofollow noreferrer">PyTorch</a> as the framework. I found this tutorial <a href="https://spandan-madan.github.io/A-Collection-of-important-tasks-in-pytorch/" rel="nofollow noreferrer">Some important Pytorch tasks - A concise summary from a vision researcher</a>, which seems to be useful.)</p>
  <blockquote>
  <ol start="2">
  <li>What are the changes I need to make while training the NN to classify one or more new classes?</li>
  </ol>
  </blockquote>
  <p>I just need to change just the number of neurons (or units) in the final layer to the number of classes/objects your new dataset contains.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-02-23T13:05:57.450Z
Answers.LastActivityDate: 2021-01-07T17:45:52.477Z
Answers.LastEditDate: 2021-01-07T17:45:52.477Z
Answers.UserDisplayName: Santhosh
Body: <p>I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework.</p>
  <p>I came across <a href="http://cs231n.github.io/transfer-learning/" rel="nofollow noreferrer">transfer learning</a>, where we fine-tune a pre-trained neural network with new data. There's a nice PyTorch tutorial explaining <a href="http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#training-the-model" rel="nofollow noreferrer">transfer learning</a>. We have a <a href="https://github.com/amdegroot/ssd.pytorch" rel="nofollow noreferrer">PyTorch implementation of the Single Shot Detector (SSD)</a> as well. See also <a href="https://towardsdatascience.com/learning-note-single-shot-multibox-detector-with-pytorch-part-1-38185e84bd79" rel="nofollow noreferrer">Single Shot MultiBox Detector with Pytorch — Part 1</a>.</p>
  <p>This is my current situation</p>
  <ul>
  <li><p>The data I want to fine-tune the neural network with is different from the data that was used to initially train the neural network; more specifically, the neural network was initially trained with a dataset of  20 classes</p>
  </li>
  <li><p>I currently have a very small labeled training dataset.</p>
  </li>
  </ul>
  <p>To solve this problem using transfer learning, the solution is to freeze the weights of the initial layers, and then train the neural network with these layers frozen.</p>
  <p>However, I am confused about what the initial layers are and how to change the last layers of the neural network to solve my specific task. So, here are my questions.</p>
  <ol>
  <li><p>What are the initial layers in this case? How exactly can I freeze them?</p>
  </li>
  <li><p>What are the changes I need to make while training the NN to classify one or more new classes?</p>
  </li>
  </ol>
  
Tags: <convolutional-neural-networks><transfer-learning><fine-tuning><single-shot-multibox-detector>
Title: When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?
UserDisplayName: Santhosh
ThreadText: # When doing transfer learning, which initial layers do we need to freeze, and how should I change the last layer for my task?
  
  (Asked by: username_0 on 2018-02-19)
  
  I want to train a neural network for the detection of a single class, but I will be extending it to detect more classes. To solve this task, I selected the PyTorch framework.
  
  I came across [transfer learning](http://cs231n.github.io/transfer-learning/), where we fine-tune a pre-trained neural network with new data. There's a nice PyTorch tutorial explaining [transfer learning](http://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html#training-the-model). We have a [PyTorch implementation of the Single Shot Detector (SSD)](https://github.com/amdegroot/ssd.pytorch) as well. See also [Single Shot MultiBox Detector with Pytorch — Part 1](https://towardsdatascience.com/learning-note-single-shot-multibox-detector-with-pytorch-part-1-38185e84bd79).
  
  This is my current situation
  
  * The data I want to fine-tune the neural network with is different from the data that was used to initially train the neural network; more specifically, the neural network was initially trained with a dataset of 20 classes
  * I currently have a very small labeled training dataset.
  
  To solve this problem using transfer learning, the solution is to freeze the weights of the initial layers, and then train the neural network with these layers frozen.
  
  However, I am confused about what the initial layers are and how to change the last layers of the neural network to solve my specific task. So, here are my questions.
  
  1. What are the initial layers in this case? How exactly can I freeze them?
  2. What are the changes I need to make while training the NN to classify one or more new classes?
  
  **Tags:** convolutional-neural-networks, transfer-learning, fine-tuning, single-shot-multibox-detector
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-02-23. Score: 1 | Accepted answer
  
  After a lot of browsing online for answers to these questions, this is what I came up with.
  
  > 1. What are the initial layers in this case? How exactly can I freeze them?
  
  The **initial few layers** are said to extract the most general features of any kind of image, like edges or corners of objects. So, I guess it actually would depend on the kind of [backbone architecture](https://www.analyticsvidhya.com/blog/2017/08/10-advanced-deep-learning-architectures-data-scientists/) you are selecting.
  
  How to freeze the layers depends on the framework we use.
  
  (I have selected [PyTorch](http://pytorch.org) as the framework. I found this tutorial [Some important Pytorch tasks - A concise summary from a vision researcher](https://spandan-madan.github.io/A-Collection-of-important-tasks-in-pytorch/), which seems to be useful.)
  
  > 2. What are the changes I need to make while training the NN to classify one or more new classes?
  
  I just need to change just the number of neurons (or units) in the final layer to the number of classes/objects your new dataset contains.
  
  
  ---
  
  