Answers.Body: <p>The approach you listed here is not really an approach, this is very very vague idea of how someone can achieve some task. You basically told we have an algorithm <code>f(image) = result</code> and there can be infinite amount of real approaches to solve this.</p>
  
  <p>In majority of CNN approaches the image travels through a convolution/pooling layers which reduces the dimensions of each current layer. In the end you end up with a significantly smaller layer which goes through the softmax and gets probabilities of different classes. This type of networks does not tell you where something was found, it just tells you that something was found somewhere in your original image.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-29T21:38:19.983Z
Answers.LastActivityDate: 2016-09-29T21:38:19.983Z
Answers.UserDisplayName: Salvador Dali
Answers.Body: <p>To add to @BlindKungFuMaster's answer, the object detection problem in NN context isn't necessarily solved by using a network that produces output with the same size as the input. Actually this approach - assigning each pixel the object probability - better fits the <a href="https://stackoverflow.com/questions/33947823/what-is-semantic-segmentation-compared-to-segmentation-and-scene-labeling">Instance/ Semantic Segmentation</a> problem. </p>
  
  <p>To briefly review the common state of the art object detectors, there are mainly two families:   </p>
  
  <ol>
  <li>The two-stage approach (one stage find the bounding boxes by a variant of sliding window, another stage finds the object probabilities, some of the layers might be common to both stages) - from <a href="https://arxiv.org/pdf/1311.2524.pdf" rel="nofollow noreferrer">RCNN</a> (propose boxes by the Selective Search algorithm, check if they contain an object by running CNN - AlexNet in this case - on each box, if a box contains an object run linear regression to tighten the box), through <a href="https://arxiv.org/pdf/1504.08083.pdf" rel="nofollow noreferrer">Fast-RCNN</a> (Share the computations for all the boxes, use ROI pooling to select the region that corresponds to each box), <a href="https://arxiv.org/pdf/1506.01497.pdf" rel="nofollow noreferrer">Faster-RCNN</a> (Speed up the region proposal stage by using a fully convolutional Region Proposal Network instead of the heavy Selective Search), <a href="https://arxiv.org/pdf/1605.06409.pdf" rel="nofollow noreferrer">R-FCN</a> (Share all computations for both objects and box proposals), and <a href="https://arxiv.org/pdf/1703.06870.pdf" rel="nofollow noreferrer">Mask-RCNN</a> that extends the faster-RCNN solution to semantic segmentation, but also solves better the detection problem by fitting the box coordinates+object features correspondence by using bilinear interpolation, replacing ROI pooling with ROI align.</li>
  <li>The single-stage approach, such as <a href="https://arxiv.org/pdf/1512.02325.pdf" rel="nofollow noreferrer">SSD</a>, <a href="https://arxiv.org/pdf/1701.06659.pdf" rel="nofollow noreferrer">DSSD</a>, <a href="https://pjreddie.com/darknet/yolo/" rel="nofollow noreferrer">YOLO</a>, <a href="https://arxiv.org/abs/1612.08242" rel="nofollow noreferrer">YOLO2</a> and lastly <a href="https://arxiv.org/pdf/1708.02002.pdf" rel="nofollow noreferrer">RetinaNet with the focal loss</a>, that in a single shot produce both boxes and objects scores, and work much faster that the two-stage approaches.  </li>
  </ol>
  
  <p>A great summary for object recognition, detection and segmentation is published <a href="https://medium.com/@nikasa1889/the-modern-history-of-object-recognition-infographic-aea18517c318" rel="nofollow noreferrer">here</a> - highly recommended.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-27T07:33:56.257Z
Answers.LastActivityDate: 2017-08-28T10:41:30.590Z
Answers.LastEditDate: 2017-08-28T10:41:30.590Z
Answers.UserDisplayName: rkellerm
Answers.Body: <p>This problem is called <a href="https://en.wikipedia.org/wiki/Object_detection" rel="noreferrer">object detection</a>.</p>
  
  <p>If you have a trainings set of images with boxed objects you can just train a neural network to directly predict the box. I.e. the output has the same dimension as the input and the NN learns to assign each pixel the probability of belonging to a certain object.</p>
  
  <p>If you don't have such a convenient dataset you could just recursively narrow the location down by feeding parts of the image to the network until you find the smallest part that still fully activates a certain classification. </p>
  
  <p><a href="https://pdfs.semanticscholar.org/713f/73ce5c3013d9fb796c21b981dc6629af0bd5.pdf" rel="noreferrer">In this paper</a> they try a mixture of these two approaches.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-09-30T12:42:03.763Z
Answers.LastActivityDate: 2016-09-30T12:42:03.763Z
Answers.UserDisplayName: BlindKungFuMaster
Body: <p>I understand how a neural network can be trained to recognise certain features in an image (faces, cars, ...), where the inputs are the image's pixels, and the output is a set of boolean values indicating which objects were recognised in the image and which weren't.</p>
  
  <p>What I don't really get is, when using this approach to detect features and we detect a face for example, how we can go back to the original image and determine the location or boundaries of the detected face. How is this achieved? Can this be achieved based on the recognition algorithm, or is a separate algorithm used to locate the face? That seems unlikely since to find the face again, it needs to be recognised in the image, which was the reason of using a NN in the first place.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-01-17T01:00:36.013Z
Comments.Text: The recent Stanford deep learning course has touched on exactly this problem: localization and detection: see, for example, [Lecture 8 Localization and Detection](https://www.youtube.com/watch?v=wFG_JMQ6_Sk&index=8&list=PLLvH2FwAQhnpj1WEB-jHmPuUeQ8mX-XXG) and [Lecture 13 - Segmentation, soft attention, spatial transformers](https://www.youtube.com/watch?v=yCrkzVFsEX0&index=13&list=PLLvH2FwAQhnpj1WEB-jHmPuUeQ8mX-XXG)
Comments.UserDisplayName: user7427549
Tags: <neural-networks><deep-learning><convolutional-neural-networks><computer-vision>
Title: When using neural networks to detect features in an image, how can locate that specific feature in the original image?
UserDisplayName: Asciiom
ThreadText: # When using neural networks to detect features in an image, how can locate that specific feature in the original image?
  
  (Asked by: username_0 on 2016-09-29)
  
  I understand how a neural network can be trained to recognise certain features in an image (faces, cars, ...), where the inputs are the image's pixels, and the output is a set of boolean values indicating which objects were recognised in the image and which weren't.
  
  What I don't really get is, when using this approach to detect features and we detect a face for example, how we can go back to the original image and determine the location or boundaries of the detected face. How is this achieved? Can this be achieved based on the recognition algorithm, or is a separate algorithm used to locate the face? That seems unlikely since to find the face again, it needs to be recognised in the image, which was the reason of using a NN in the first place.
  
  **Tags:** neural-networks, deep-learning, convolutional-neural-networks, computer-vision
  
  > **Comments:**
  > **username_1**: The recent Stanford deep learning course has touched on exactly this problem: localization and detection: see, for example, [Lecture 8 Localization and Detection](https://www.youtube.com/watch?v=wFG_JMQ6_Sk&index=8&list=PLLvH2FwAQhnpj1WEB-jHmPuUeQ8mX-XXG) and [Lecture 13 - Segmentation, soft attention, spatial transformers](https://www.youtube.com/watch?v=yCrkzVFsEX0&index=13&list=PLLvH2FwAQhnpj1WEB-jHmPuUeQ8mX-XXG)  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_4 on 2016-09-30. Score: 9 | Accepted answer
  
  This problem is called [object detection](https://en.wikipedia.org/wiki/Object_detection).
  
  If you have a trainings set of images with boxed objects you can just train a neural network to directly predict the box. I.e. the output has the same dimension as the input and the NN learns to assign each pixel the probability of belonging to a certain object.
  
  If you don't have such a convenient dataset you could just recursively narrow the location down by feeding parts of the image to the network until you find the smallest part that still fully activates a certain classification.
  
  [In this paper](https://pdfs.semanticscholar.org/713f/73ce5c3013d9fb796c21b981dc6629af0bd5.pdf) they try a mixture of these two approaches.
  
  
  ---
  
  ### Answer by username_2 on 2016-09-29. Score: 3
  
  The approach you listed here is not really an approach, this is very very vague idea of how someone can achieve some task. You basically told we have an algorithm `f(image) = result` and there can be infinite amount of real approaches to solve this.
  
  In majority of CNN approaches the image travels through a convolution/pooling layers which reduces the dimensions of each current layer. In the end you end up with a significantly smaller layer which goes through the softmax and gets probabilities of different classes. This type of networks does not tell you where something was found, it just tells you that something was found somewhere in your original image.
  
  
  ---
  
  ### Answer by username_3 on 2017-08-27. Score: 2
  
  To add to @username_4's answer, the object detection problem in NN context isn't necessarily solved by using a network that produces output with the same size as the input. Actually this approach - assigning each pixel the object probability - better fits the [Instance/ Semantic Segmentation](https://stackoverflow.com/questions/33947823/what-is-semantic-segmentation-compared-to-segmentation-and-scene-labeling) problem.
  
  To briefly review the common state of the art object detectors, there are mainly two families:
  
  1. The two-stage approach (one stage find the bounding boxes by a variant of sliding window, another stage finds the object probabilities, some of the layers might be common to both stages) - from [RCNN](https://arxiv.org/pdf/1311.2524.pdf) (propose boxes by the Selective Search algorithm, check if they contain an object by running CNN - AlexNet in this case - on each box, if a box contains an object run linear regression to tighten the box), through [Fast-RCNN](https://arxiv.org/pdf/1504.08083.pdf) (Share the computations for all the boxes, use ROI pooling to select the region that corresponds to each box), [Faster-RCNN](https://arxiv.org/pdf/1506.01497.pdf) (Speed up the region proposal stage by using a fully convolutional Region Proposal Network instead of the heavy Selective Search), [R-FCN](https://arxiv.org/pdf/1605.06409.pdf) (Share all computations for both objects and box proposals), and [Mask-RCNN](https://arxiv.org/pdf/1703.06870.pdf) that extends the faster-RCNN solution to semantic segmentation, but also solves better the detection problem by fitting the box coordinates+object features correspondence by using bilinear interpolation, replacing ROI pooling with ROI align.
  2. The single-stage approach, such as [SSD](https://arxiv.org/pdf/1512.02325.pdf), [DSSD](https://arxiv.org/pdf/1701.06659.pdf), [YOLO](https://pjreddie.com/darknet/yolo/), [YOLO2](https://arxiv.org/abs/1612.08242) and lastly [RetinaNet with the focal loss](https://arxiv.org/pdf/1708.02002.pdf), that in a single shot produce both boxes and objects scores, and work much faster that the two-stage approaches.
  
  A great summary for object recognition, detection and segmentation is published [here](https://medium.com/@nikasa1889/the-modern-history-of-object-recognition-infographic-aea18517c318) - highly recommended.
  
  
  ---
  
  