Answers.Body: <p>Yes, there are a few smarter ways to add a new class to your pre-trained model without retraining the entire network from scratch. Here are some strategies:</p>
  <p><strong>Fine-tuning the last layer</strong></p>
  <p>Instead of retraining the whole model, you can:</p>
  <p>Replace the final fully connected layer (softmax) to accommodate 11 classes.
  Freeze all the layers except the last few and fine-tune only those layers. This can significantly reduce the training time while maintaining much of the learned knowledge from the previous training.</p>
  <p>To do this you can follow this steps</p>
  <p>=&gt; Replace the output layer with one that has 11 neurons (for the new class).</p>
  <p>=&gt; Freeze the earlier (convolutional layers) and train only the new final layer and possibly the penultimate fully connected layer.</p>
  <p>=&gt; Use the new dataset (with the 11th class added) to fine-tune the network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-24T21:44:31.930Z
Answers.Text: I will give this method a try
Answers.UserDisplayName: Yauhen Yakimenka
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-08T19:45:31.417Z
Answers.Text: Yes, this gives a good performance, much less epochs needed and each epoch is now faster (as there are less parameters to train). I am thinking also to try to freeze ALL the weights except for the newly added (corresponding to the new 11th output node). This cannot be done in one layer in PyTorch I guess (the whole layer can be frozen or not), so I need to create and ad-hoc architecture for the last layer. I am curious to see the results
Answers.UserDisplayName: Yauhen Yakimenka
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-09-24T21:13:17.590Z
Answers.LastActivityDate: 2024-09-24T21:13:17.590Z
Answers.UserDisplayName: Keval Pandya
Answers.Body: <p>If I understood your problem correctly, you might try the following.</p>
  <p>Relabel the first data set (of 10 classes) as one class 'A' and label the new set as class 'B'. Merge two sets and train a two-class NNET.</p>
  <p>In test mode, apply first the new 2-class classifier and collect all points that it declared class 'A'. Then use the already trained 10-class classifier on these points.</p>
  <p>The performance is anybody's guess at this point, so it would be interesting to see your actual results.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-08T19:47:12.553Z
Answers.Text: It's an interesting idea. But the resulting architecture won't be a single NN but a pipeline of those two. This bothers me aesthetically :)
Answers.UserDisplayName: Yauhen Yakimenka
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-10-07T17:47:50.197Z
Answers.LastActivityDate: 2024-10-07T17:47:50.197Z
Answers.UserDisplayName: dnqxt
Body: <p>I have a dataset of 10 types of images. I trained AlexNet to classify these images, and it gives decent results (around 80% accuracy).</p>
  <p>Now, I want to add the 11th class of images. Of course, I can change the last (softmax) layer to have the output size of 11 and retrained on the whole combined dataset of 11 classes of images. But is there a smarter way to do that, provided that I already have a pre-trained CNN?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-09-24T20:13:40.770Z
Comments.Text: depdns what you mean with smarter way, you won't get away without additional parameters, but you can just add a new set of parameters to the last layer, and train only that one (though it will use features about the first 10 classes to predict the 11th, so probably suboptimal)
Comments.UserDisplayName: Alberto
Tags: <machine-learning><convolutional-neural-networks><classification>
Title: Adding one new class of images to the dataset and re-training the classifier model
UserDisplayName: Yauhen Yakimenka
ThreadText: # Adding one new class of images to the dataset and re-training the classifier model
  
  (Asked by: username_0 on 2024-09-24)
  
  I have a dataset of 10 types of images. I trained AlexNet to classify these images, and it gives decent results (around 80% accuracy).
  
  Now, I want to add the 11th class of images. Of course, I can change the last (softmax) layer to have the output size of 11 and retrained on the whole combined dataset of 11 classes of images. But is there a smarter way to do that, provided that I already have a pre-trained CNN?
  
  **Tags:** machine-learning, convolutional-neural-networks, classification
  > **Comments:**
  > **username_1**: depdns what you mean with smarter way, you won't get away without additional parameters, but you can just add a new set of parameters to the last layer, and train only that one (though it will use features about the first 10 classes to predict the 11th, so probably suboptimal)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2024-09-24. Score: 1 | Accepted answer
  
  Yes, there are a few smarter ways to add a new class to your pre-trained model without retraining the entire network from scratch. Here are some strategies:
  
  **Fine-tuning the last layer**
  
  Instead of retraining the whole model, you can:
  
  Replace the final fully connected layer (softmax) to accommodate 11 classes.
  Freeze all the layers except the last few and fine-tune only those layers. This can significantly reduce the training time while maintaining much of the learned knowledge from the previous training.
  
  To do this you can follow this steps
  
  => Replace the output layer with one that has 11 neurons (for the new class).
  
  => Freeze the earlier (convolutional layers) and train only the new final layer and possibly the penultimate fully connected layer.
  
  => Use the new dataset (with the 11th class added) to fine-tune the network.
  
  > **Comments:**
  > **username_0**: I will give this method a try  
  > **username_0**: Yes, this gives a good performance, much less epochs needed and each epoch is now faster (as there are less parameters to train). I am thinking also to try to freeze ALL the weights except for the newly added (corresponding to the new 11th output node). This cannot be done in one layer in PyTorch I guess (the whole layer can be frozen or not), so I need to create and ad-hoc architecture for the last layer. I am curious to see the results  
  
  ---
  
  ### Answer by username_3 on 2024-10-07. Score: 2
  
  If I understood your problem correctly, you might try the following.
  
  Relabel the first data set (of 10 classes) as one class 'A' and label the new set as class 'B'. Merge two sets and train a two-class NNET.
  
  In test mode, apply first the new 2-class classifier and collect all points that it declared class 'A'. Then use the already trained 10-class classifier on these points.
  
  The performance is anybody's guess at this point, so it would be interesting to see your actual results.
  
  > **Comments:**
  > **username_0**: It's an interesting idea. But the resulting architecture won't be a single NN but a pipeline of those two. This bothers me aesthetically :)  
  
  ---
  
  