Answers.Body: <p>I have put my initial idea to a test and used a small pretrained CNN (MobileNet) to compute features for reference images and stored the feature vectors in a &quot;database&quot;. Query images go through the exact same network and the resulting feature vector is used for nearest neighbor retrieval in the DB.</p>
  <pre><code>from glob import glob
  
  import torch
  from PIL import Image
  from numpy.linalg import norm
  from torchvision import transforms
  from torchvision.models import mobilenet_v2
  
  model = mobilenet_v2(pretrained=True)
  model.eval()
  
  preprocess = transforms.Compose([
      transforms.Resize(256),
      transforms.CenterCrop(224),
      transforms.ToTensor(),
      transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
  ])
  
  # Generate DB
  db = {}
  dp_paths = glob('db/*.jpg')
  for path in dp_paths:
      image = preprocess(Image.open(path)).unsqueeze(0)
      with torch.no_grad():
          output = model(image)
      db[output] = path
  
  # Query
  image = preprocess(Image.open('queries/box.jpg')).unsqueeze(0)
  with torch.no_grad():
      query = model(image)
  
  # Nearest Neighbor (poor man's version)
  min_distance = float('inf')
  candidate = None
  for k, v in db.items():
      distance = norm(k.numpy() - query.numpy())
      if distance &lt; min_distance:
          min_distance = distance
          candidate = v
  
  print(candidate, min_distance)
  </code></pre>
  <p>At least with my 5 test reference images and several query images it worked without a single failed &quot;classification&quot;. However, I am not sure if it will stand up to a larger test...</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-01T12:47:53.917Z
Answers.LastActivityDate: 2020-11-01T13:00:54.370Z
Answers.LastEditDate: 2020-11-01T13:00:54.370Z
Answers.UserDisplayName: sonovice
Body: <p>I am looking for a way to re-identify/classify/recognize x real life objects (x &lt; 50) with a camera. Each object should be presented to the AI only once for learning and there's always only one of these objects in the query image. New objects should be addable to the list of &quot;known&quot; objects. The objects are not necessarily part of ImageNet nor do I have a training dataset with various instances of these objects.</p>
  <p>Example:</p>
  <blockquote>
  <p>In the beginning I have no &quot;known&quot; objects. Now I present a
  smartphone, a teddy bear and a pair of scissors to the system. It
  should learn to re-identify these three objects if presented in the
  future. The objects will be the exact same objects, i.e. not a different phone, but definitely in a different viewing angle, lighting etc.</p>
  </blockquote>
  <p>My understanding is that I would have to place each object in an embedding space and do a simple nearest neighbor lookup in that space for the queries. Maybe just use a trained ResNet, cut off the classification and simply use the output vector for each object? Not sure what the best way would be.</p>
  <p>Any advice or hint to the right direction would be highly appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-31T17:43:56.153Z
Comments.Text: This is basically not possible.
Comments.UserDisplayName: FourierFlux
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-31T18:06:25.130Z
Comments.Text: I have heard that exact sentence a couple of times in the past. Most times there was a solution after all, so I am not giving up so fast.
  One idea that comes to mind: Learn distinctive features of objects: Fuzziness, main color, size etc... But that would also need a dataset or at least another annotation pass for all the ImageNet classes.
Comments.UserDisplayName: sonovice
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-10-31T22:58:33.750Z
Comments.Text: The problem is to don't get a 3d view of the object, conceptually for rigid objects it you get a 3d image of it you can do a brute force search.
Comments.UserDisplayName: FourierFlux
Tags: <deep-learning><convolutional-neural-networks><classification><object-recognition><one-shot-learning>
Title: Single-Shot Learning for Object Re-Identification
UserDisplayName: sonovice
ThreadText: # Single-Shot Learning for Object Re-Identification
  
  (Asked by: username_0 on 2020-10-31)
  
  I am looking for a way to re-identify/classify/recognize x real life objects (x < 50) with a camera. Each object should be presented to the AI only once for learning and there's always only one of these objects in the query image. New objects should be addable to the list of "known" objects. The objects are not necessarily part of ImageNet nor do I have a training dataset with various instances of these objects.
  
  Example:
  
  > In the beginning I have no "known" objects. Now I present a
  > smartphone, a teddy bear and a pair of scissors to the system. It
  > should learn to re-identify these three objects if presented in the
  > future. The objects will be the exact same objects, i.e. not a different phone, but definitely in a different viewing angle, lighting etc.
  
  My understanding is that I would have to place each object in an embedding space and do a simple nearest neighbor lookup in that space for the queries. Maybe just use a trained ResNet, cut off the classification and simply use the output vector for each object? Not sure what the best way would be.
  
  Any advice or hint to the right direction would be highly appreciated.
  
  **Tags:** deep-learning, convolutional-neural-networks, classification, object-recognition, one-shot-learning
  
  > **Comments:**
  > **username_1**: This is basically not possible.  
  > **username_0**: I have heard that exact sentence a couple of times in the past. Most times there was a solution after all, so I am not giving up so fast. One idea that comes to mind: Learn distinctive features of objects: Fuzziness, main color, size etc... But that would also need a dataset or at least another annotation pass for all the ImageNet classes.  
  > **username_1**: The problem is to don't get a 3d view of the object, conceptually for rigid objects it you get a 3d image of it you can do a brute force search.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-11-01. Score: 0 | Accepted answer
  
  I have put my initial idea to a test and used a small pretrained CNN (MobileNet) to compute features for reference images and stored the feature vectors in a "database". Query images go through the exact same network and the resulting feature vector is used for nearest neighbor retrieval in the DB.
  
  ```
  from glob import glob
  
  import torch
  from PIL import Image
  from numpy.linalg import norm
  from torchvision import transforms
  from torchvision.models import mobilenet_v2
  
  model = mobilenet_v2(pretrained=True)
  model.eval()
  
  preprocess = transforms.Compose([
      transforms.Resize(256),
      transforms.CenterCrop(224),
      transforms.ToTensor(),
      transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
  ])
  
  # Generate DB
  db = {}
  dp_paths = glob('db/*.jpg')
  for path in dp_paths:
      image = preprocess(Image.open(path)).unsqueeze(0)
      with torch.no_grad():
          output = model(image)
      db[output] = path
  
  # Query
  image = preprocess(Image.open('queries/box.jpg')).unsqueeze(0)
  with torch.no_grad():
      query = model(image)
  
  # Nearest Neighbor (poor man's version)
  min_distance = float('inf')
  candidate = None
  for k, v in db.items():
      distance = norm(k.numpy() - query.numpy())
      if distance < min_distance:
          min_distance = distance
          candidate = v
  
  print(candidate, min_distance)
  
  ```
  
  At least with my 5 test reference images and several query images it worked without a single failed "classification". However, I am not sure if it will stand up to a larger test...
  
  
  ---
  
  