Answers.Body: <p>There are many different problems in computer vision. Four of them are well described by the <a href="https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852" rel="nofollow noreferrer">top image here</a>:</p>
  
  <ul>
  <li>Classification: Given an image, say what is on it (a single thing)</li>
  <li>Classification+Localization: Given an image, say what is on it and draw an axis-aligned bounding box (AABB) around it</li>
  <li>Object detection: Given an image, draw AABBs around every object and classify those objects</li>
  <li>Semantic segmentation: See <a href="https://arxiv.org/pdf/1602.06541.pdf" rel="nofollow noreferrer">Survey of semantic segmentation</a></li>
  <li>Instance segmentatation: Like semantic segmentation, but if there are multiple cats then they should be recognized as different objects.</li>
  </ul>
  
  <p>Your question seems to be about object detection. The relevant papers here are:</p>
  
  <ul>
  <li><a href="https://arxiv.org/pdf/1512.02325.pdf" rel="nofollow noreferrer">SSD: Single Shot MultiBox Detector</a>: <a href="https://martin-thoma.com/object-detection/" rel="nofollow noreferrer">Code</a></li>
  <li><a href="https://arxiv.org/abs/1506.02640v5" rel="nofollow noreferrer">You Only Look Once: Unified, Real-Time Object Detection</a> (YOLO): <a href="https://pjreddie.com/darknet/yolo/" rel="nofollow noreferrer">Website with video</a></li>
  <li><a href="https://arxiv.org/pdf/1804.02767.pdf" rel="nofollow noreferrer">YOLOv3</a></li>
  </ul>
  
  <p>If you actually already have the regions, then you can simply perform classification on them. When you pad / scale / crop them, you can batch-predict them.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-24T06:41:32.140Z
Answers.LastActivityDate: 2018-09-24T06:41:32.140Z
Answers.UserDisplayName: Martin Thoma
Body: <p>Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision application and are used widely in the industry, for example for internet search engines and autonomous cars. In real life problems, the image contains of regions with different objects. It is not enough to only identify the picture but elements of the picture.</p>
  
  <p>A while ago an alternative to the well known sliding window algorithm was described in the literature, called Region Proposal Networks. It is basically a convolution neural network which was extended by a region vector.</p>
  
  <p><strong>Problem that I am trying to solve:</strong></p>
  
  <p>In a given video frame, I want to pick some region of interests (literally), and perform classification on those regions.</p>
  
  <p><strong>How is it currently implemented</strong></p>
  
  <ol>
  <li>Capture the video frame</li>
  <li>Split the video frame into multiple images each representing a region of interest</li>
  <li>Perform image classification(inference) on each of the image (corresponding to a part of the frame)</li>
  <li>Aggregate the results of #3 </li>
  </ol>
  
  <p><strong>Problem with the current approach</strong></p>
  
  <p>Multiple inferences per frame.</p>
  
  <p><strong>Question</strong></p>
  
  <p>I am looking for a solution where I specify the locations of interest in a frame, and inference task, be it object detection (or) image classification, is performed only on those regions.Can you please point to me the references which I need to study (or) use to do this.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-08-24T09:53:26.570Z
Comments.Text: Is your requirement to supply your own "region of interest" data fixed? That would rule out architectures like YOLO, which do very close to what you want overall, but internally decide on locations.
Comments.UserDisplayName: Neil Slater
Tags: <deep-learning><classification><computer-vision><object-recognition>
Title: Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)
UserDisplayName: Vivek Maran
ThreadText: # Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame)
  
  (Asked by: username_0 on 2018-08-24)
  
  Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision application and are used widely in the industry, for example for internet search engines and autonomous cars. In real life problems, the image contains of regions with different objects. It is not enough to only identify the picture but elements of the picture.
  
  A while ago an alternative to the well known sliding window algorithm was described in the literature, called Region Proposal Networks. It is basically a convolution neural network which was extended by a region vector.
  
  **Problem that I am trying to solve:**
  
  In a given video frame, I want to pick some region of interests (literally), and perform classification on those regions.
  
  **How is it currently implemented**
  
  1. Capture the video frame
  2. Split the video frame into multiple images each representing a region of interest
  3. Perform image classification(inference) on each of the image (corresponding to a part of the frame)
  4. Aggregate the results of #3
  
  **Problem with the current approach**
  
  Multiple inferences per frame.
  
  **Question**
  
  I am looking for a solution where I specify the locations of interest in a frame, and inference task, be it object detection (or) image classification, is performed only on those regions.Can you please point to me the references which I need to study (or) use to do this.
  
  **Tags:** deep-learning, classification, computer-vision, object-recognition
  > **Comments:**
  > **username_1**: Is your requirement to supply your own "region of interest" data fixed? That would rule out architectures like YOLO, which do very close to what you want overall, but internally decide on locations.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-09-24. Score: 1
  
  There are many different problems in computer vision. Four of them are well described by the [top image here](https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852):
  
  * Classification: Given an image, say what is on it (a single thing)
  * Classification+Localization: Given an image, say what is on it and draw an axis-aligned bounding box (AABB) around it
  * Object detection: Given an image, draw AABBs around every object and classify those objects
  * Semantic segmentation: See [Survey of semantic segmentation](https://arxiv.org/pdf/1602.06541.pdf)
  * Instance segmentatation: Like semantic segmentation, but if there are multiple cats then they should be recognized as different objects.
  
  Your question seems to be about object detection. The relevant papers here are:
  
  * [SSD: Single Shot MultiBox Detector](https://arxiv.org/pdf/1512.02325.pdf): [Code](https://martin-thoma.com/object-detection/)
  * [You Only Look Once: Unified, Real-Time Object Detection](https://arxiv.org/abs/1506.02640v5) (YOLO): [Website with video](https://pjreddie.com/darknet/yolo/)
  * [YOLOv3](https://arxiv.org/pdf/1804.02767.pdf)
  
  If you actually already have the regions, then you can simply perform classification on them. When you pad / scale / crop them, you can batch-predict them.
  
  
  ---
  
  