Answers.Body: <p>In my experience, if you have really discriminative objects with distinct features then yes! The original DeepSORT's reid model can be borrowed to track those things.
  But for the best result, you should train your own reid model to suit the task and the classes that you want to track.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-30T21:55:02.990Z
Answers.Text: By DeepSORT's original reid model do you mean the default `resources/networks/mars-small128.pb`?
Answers.UserDisplayName: Mehdi Charife
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-30T21:58:33.757Z
Answers.Text: Yes, that's correct :D
Answers.UserDisplayName: GunFire
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-30T22:00:00.340Z
Answers.Text: Isn't that model only trained for extracting people features, and thus won't work on other objects?
Answers.UserDisplayName: Mehdi Charife
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-30T22:06:23.903Z
Answers.Text: Theoretically, yes. But in reality, it CAN BE BORROWED because kernel filters that learned  to extract people feature can still have different reactions to different features from unseen data. From those information, model might can still know something and patchworkly embbed those features in its embedding space.
Answers.UserDisplayName: GunFire
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-31T19:02:01.050Z
Answers.Text: You said that I should train my own reid model to have better results. Do you have something that can get me started? I'm relatively new to this and so I'm basically lost when it comes to what is normally done in these cases.
Answers.UserDisplayName: Mehdi Charife
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-01T05:20:51.140Z
Answers.Text: Sure, at first if you are new to this field, you can try re-train the base OSNet model. 
  
  For the next step, you could try replacing the backbone in the architecture with newer backbone for better feature extraction.
  
  Furthermore, you can read related papers to understand better about the reid task, I would recommend you read more about metric - contrastive learning (Arcface, SubArcface, InfoNCE,...) and some training strategies (self-supervised, SupCon,...). 
  
  Finally, learn and learn more. Expand your knowledge base so you can come up with many ideas to work on and improve your result.
Answers.UserDisplayName: GunFire
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-30T21:51:48.517Z
Answers.LastActivityDate: 2023-05-30T21:51:48.517Z
Answers.UserDisplayName: GunFire
Body: <p>As far as my understanding goes, the model used for feature extraction in <a href="https://github.com/nwojke/deep_sort" rel="nofollow noreferrer">DeepSort</a> is specified as the first argument of the function <code>create_box_encoder</code> in the file <a href="https://github.com/nwojke/deep_sort/blob/master/tools/generate_detections.py" rel="nofollow noreferrer"><code>tools/generate_detections.py</code></a>:</p>
  <pre><code>def create_box_encoder(model_filename, input_name=&quot;images&quot;,
                         output_name=&quot;features&quot;, batch_size=32):
      image_encoder = ImageEncoder(model_filename, input_name, output_name)
      image_shape = image_encoder.image_shape
  
      def encoder(image, boxes):
          image_patches = []
          for box in boxes:
              patch = extract_image_patch(image, box, image_shape[:2])
              if patch is None:
                  print(&quot;WARNING: Failed to extract image patch: %s.&quot; % str(box))
                  patch = np.random.uniform(
                      0., 255., image_shape).astype(np.uint8)
              image_patches.append(patch)
          image_patches = np.asarray(image_patches)
          return image_encoder(image_patches, batch_size)
  
      return encoder
  </code></pre>
  <p>In the same file, the default value of the argument <code>model_filename</code> is specified under the <code>parse_args()</code> function to be <code>resources/networks/mars-small128.pb</code>, which appears to be a model for person re-identification.</p>
  <p>Can a model for re-identifying objects other than people (and from multiple classes, such as cars, birds, trucks, etc) be used instead in DeepSort? If so, does DeepSort provide any means for training such models?</p>
  <p>My initial understanding was that DeepSort would be able to track all classes recognized by a trained YOLO model. I didn't know that a stand-alone feature extractor was required.</p>
  
Tags: <computer-vision><object-detection><yolo><feature-extraction><object-tracking>
Title: Can DeepSort be made to track objects beside people?
UserDisplayName: Mehdi Charife
ThreadText: # Can DeepSort be made to track objects beside people?
  
  (Asked by: username_0 on 2023-05-30)
  
  As far as my understanding goes, the model used for feature extraction in [DeepSort](https://github.com/nwojke/deep_sort) is specified as the first argument of the function `create_box_encoder` in the file [`tools/generate_detections.py`](https://github.com/nwojke/deep_sort/blob/master/tools/generate_detections.py):
  
  ```
  def create_box_encoder(model_filename, input_name="images",
                         output_name="features", batch_size=32):
      image_encoder = ImageEncoder(model_filename, input_name, output_name)
      image_shape = image_encoder.image_shape
  
      def encoder(image, boxes):
          image_patches = []
          for box in boxes:
              patch = extract_image_patch(image, box, image_shape[:2])
              if patch is None:
                  print("WARNING: Failed to extract image patch: %s." % str(box))
                  patch = np.random.uniform(
                      0., 255., image_shape).astype(np.uint8)
              image_patches.append(patch)
          image_patches = np.asarray(image_patches)
          return image_encoder(image_patches, batch_size)
  
      return encoder
  
  ```
  
  In the same file, the default value of the argument `model_filename` is specified under the `parse_args()` function to be `resources/networks/mars-small128.pb`, which appears to be a model for person re-identification.
  
  Can a model for re-identifying objects other than people (and from multiple classes, such as cars, birds, trucks, etc) be used instead in DeepSort? If so, does DeepSort provide any means for training such models?
  
  My initial understanding was that DeepSort would be able to track all classes recognized by a trained YOLO model. I didn't know that a stand-alone feature extractor was required.
  
  **Tags:** computer-vision, object-detection, yolo, feature-extraction, object-tracking
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-05-30. Score: 2 | Accepted answer
  
  In my experience, if you have really discriminative objects with distinct features then yes! The original DeepSORT's reid model can be borrowed to track those things.
  But for the best result, you should train your own reid model to suit the task and the classes that you want to track.
  
  > **Comments:**
  > **username_0**: By DeepSORT's original reid model do you mean the default `resources/networks/mars-small128.pb`?  
  > **username_1**: Yes, that's correct :D  
  > **username_0**: Isn't that model only trained for extracting people features, and thus won't work on other objects?  
  > **username_1**: Theoretically, yes. But in reality, it CAN BE BORROWED because kernel filters that learned  to extract people feature can still have different reactions to different features from unseen data. From those information, model might can still know something and patchworkly embbed those features in its embedding space.  
  > **username_0**: You said that I should train my own reid model to have better results. Do you have something that can get me started? I'm relatively new to this and so I'm basically lost when it comes to what is normally done in these cases.  
  > **username_1**: Sure, at first if you are new to this field, you can try re-train the base OSNet model.   For the next step, you could try replacing the backbone in the architecture with newer backbone for better feature extraction.  Furthermore, you can read related papers to understand better about the reid task, I would recommend you read more about metric - contrastive learning (Arcface, SubArcface, InfoNCE,...) and some training strategies (self-supervised, SupCon,...).   Finally, learn and learn more. Expand your knowledge base so you can come up with many ideas to work on and improve your result.  
  
  ---
  
  