Answers.Body: <p>I have one possible solution for you inspired by real-time face recognition system. It is a similar case to you.</p>
  <ol>
  <li>Create embeddings for each class i.e. person or in your case, a class. (Using Siamese network with ArcFace loss)</li>
  <li>When a new image comes, take L1 or cosine distance with the embeddings. You will need a threshold above which you create a embedding for the new guy and give it a new class.</li>
  </ol>
  <p>One issue you will think at this point of time is how do you do that when have billions of data points. The search for the matching embeddings can then be done using Hierarchical Navigable Small World graphs (see <a href="https://github.com/nmslib/hnswlib" rel="nofollow noreferrer">https://github.com/nmslib/hnswlib</a>).</p>
  <p>This solution is loosely connected to the plausible techniques you have mentioned here. And if you still  want something in that regard, I shall add the keyword Active Learning.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-12T08:37:25.197Z
Answers.Text: Thanks for suggesting facial recognition based approach. I have thought the same but input images are very noisy, say lot of extra things are written on the image(noise) before scan, so I was thinking that Face recog based approach will have hard time finding similarity. ON top of that I have many classes in which image can go, so timing is also a problem with face recog based approach. I can compromise with time issue but not sure if 1st issue(noise) can be taken care by Siamese based network.
Answers.UserDisplayName: Rambo_john
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-13T02:57:08.867Z
Answers.Text: Try cleaning your images with a GAN or VAE based approach and then move on with this one. Apart from that, it will be a good idea to give this a try since facial recognition system really get a lot of different kind of noises still go on.
Answers.UserDisplayName: Abhishek Verma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-13T16:11:26.380Z
Answers.Text: Many thanks for VAN /GAN based cleaning approach suggestion. Will try first FR based and then will go for GAN/VAN based approach.
Answers.UserDisplayName: Rambo_john
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-10T23:15:49.307Z
Answers.LastActivityDate: 2021-02-10T23:15:49.307Z
Answers.UserDisplayName: Abhishek Verma
Answers.Body: <p>Matching network and prototypical network few-shot learning can be a better option than Siamese network, There are many implementation available online. Also, these methods are fast in computation or classification.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-23T23:52:34.433Z
Answers.LastActivityDate: 2021-11-24T07:40:08.727Z
Answers.LastEditDate: 2021-11-24T07:40:08.727Z
Answers.UserDisplayName: Rambo_john
Body: <p>I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new classes of scanned images can be added and the model should not only classify them as unseen (open set image recognition), but it should be able to tell in which new class it should belong (not able to figure out the implementation for this.)</p>
  <p>So, I am thinking that the below option can work for the classification of unseen classes</p>
  <ol>
  <li><p>Zero-shot learning: Once the image is classified as unseen, we can then apply zero-shot learning to find its respective class for sorting.</p>
  </li>
  <li><p>Template matching: Match the test image of unseen classes with all available class images, and, once we have a match, we can do sorting of images.</p>
  </li>
  <li><p>Meta learning-based approach: I am not sure how to implement this, suggestions are much appreciated.</p>
  </li>
  </ol>
  <p>Note: I already tried the classical computer vision approach, but it's not working out. So, more open for neural net-based approach.</p>
  <p>Is my approach to solving the problem correct? If possible, suggest some alternative to find the corresponding match/classification of the unseen class image. As I could think of these 2 alternative solutions only.</p>
  
Tags: <neural-networks><classification><representation-learning><active-learning><zero-shot-learning>
Title: classification of unseen classes of image in open set classification
UserDisplayName: Rambo_john
ThreadText: # classification of unseen classes of image in open set classification
  
  (Asked by: username_0 on 2021-02-09)
  
  I have a scanned image, and they need to be classified in one of the pre-defined image classes, so that it can be sorted. However, the problem is the open nature of the classes. At testing time, new classes of scanned images can be added and the model should not only classify them as unseen (open set image recognition), but it should be able to tell in which new class it should belong (not able to figure out the implementation for this.)
  
  So, I am thinking that the below option can work for the classification of unseen classes
  
  1. Zero-shot learning: Once the image is classified as unseen, we can then apply zero-shot learning to find its respective class for sorting.
  2. Template matching: Match the test image of unseen classes with all available class images, and, once we have a match, we can do sorting of images.
  3. Meta learning-based approach: I am not sure how to implement this, suggestions are much appreciated.
  
  Note: I already tried the classical computer vision approach, but it's not working out. So, more open for neural net-based approach.
  
  Is my approach to solving the problem correct? If possible, suggest some alternative to find the corresponding match/classification of the unseen class image. As I could think of these 2 alternative solutions only.
  
  **Tags:** neural-networks, classification, representation-learning, active-learning, zero-shot-learning
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2021-02-10. Score: 0
  
  I have one possible solution for you inspired by real-time face recognition system. It is a similar case to you.
  
  1. Create embeddings for each class i.e. person or in your case, a class. (Using Siamese network with ArcFace loss)
  2. When a new image comes, take L1 or cosine distance with the embeddings. You will need a threshold above which you create a embedding for the new guy and give it a new class.
  
  One issue you will think at this point of time is how do you do that when have billions of data points. The search for the matching embeddings can then be done using Hierarchical Navigable Small World graphs (see <https://github.com/nmslib/hnswlib>).
  
  This solution is loosely connected to the plausible techniques you have mentioned here. And if you still want something in that regard, I shall add the keyword Active Learning.
  
  > **Comments:**
  > **username_0**: Thanks for suggesting facial recognition based approach. I have thought the same but input images are very noisy, say lot of extra things are written on the image(noise) before scan, so I was thinking that Face recog based approach will have hard time finding similarity. ON top of that I have many classes in which image can go, so timing is also a problem with face recog based approach. I can compromise with time issue but not sure if 1st issue(noise) can be taken care by Siamese based network.  
  > **username_1**: Try cleaning your images with a GAN or VAE based approach and then move on with this one. Apart from that, it will be a good idea to give this a try since facial recognition system really get a lot of different kind of noises still go on.  
  > **username_0**: Many thanks for VAN /GAN based cleaning approach suggestion. Will try first FR based and then will go for GAN/VAN based approach.  
  
  ---
  
  ### Answer by username_0 on 2021-11-23. Score: 0
  
  Matching network and prototypical network few-shot learning can be a better option than Siamese network, There are many implementation available online. Also, these methods are fast in computation or classification.
  
  
  ---
  
  