Answers.Body: <p>I think you're describing "object localization and detection" which combines object identification with discovery of its spatial placement in the field of view.  There's been a lot of work on this in the past 5 years using CNNs and variants.  It's very much in demand for pedestrian/obstruction detection, identification, and avoidance in autonomous car systems.</p>
  
  <p>Here's a place to start:  <a href="https://github.com/kjw0612/awesome-deep-vision" rel="nofollow noreferrer">https://github.com/kjw0612/awesome-deep-vision</a></p>
  
  <p>Especially, take a look at "Faster R-CNN" by Ren et al, 2016.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-22T20:52:37.127Z
Answers.LastActivityDate: 2017-05-22T20:52:37.127Z
Answers.UserDisplayName: Randy
Body: <p>I understand that there are flavors of (convolutional) neural networks that are useful for object <a href="https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/" rel="nofollow noreferrer">localization and detection</a> tasks of reasonable difficulty. In all of the examples I have seen so far, localization is formulated as finding the corners of a bounding box. Often, the fit is not expected to be very precise:</p>
  
  <p><a href="https://i.sstatic.net/xmWck.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/xmWck.jpg" alt="traffic sign bounding box"></a></p>
  
  <p>Conversely, I am interested in a task I want to achieve very precise localization and characterization of some simple shapes or objects. As an example of one of the simplest cases I can think of, my inputs will be images like the following:</p>
  
  <p><a href="https://i.sstatic.net/NOqN5.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NOqN5.jpg" alt="28 21 15"></a></p>
  
  <p>Given this 60x60 image, I want my neural net(s), via <strong>regression</strong>, to tell me that the circle's <strong>diameter is 18px</strong> and its <strong>centre is located at (28, 21)</strong> from top left. (I will train it using similar 60x60 images with white circles of various sizes on black backgrounds.)</p>
  
  <p>Later I am interested in dealing with similar tasks in the real world, e.g. spheres/cubes/cylinders with different viewing angles, light conditions, occlusions, etc. However, I am interested in solving this simplest case first. (One reason is that I can generate this data very easily.)</p>
  
  <p>I have the following specific questions:</p>
  
  <ol>
  <li>Has anyone used neural nets for this sort of tasks before? (e.g. precisely determining sizes and centroids of objects)</li>
  <li>My understanding is that these things are at least theoretically possible using convolutional nets, or even sufficiently complicated vanilla fully connected nets. Is this correct? </li>
  <li>What architecture(s) would be appropriate for these tasks?</li>
  </ol>
  
  <p>Note: I am aware that fitting a bounding box to the circles and calculating its centre and size will solve this particular case, but it will not generalize to handle occlusions, changing lights, etc. I would like to move towards a method which can, for example, calculate the centroids and diameters of spheres in real-world B&amp;W photos.</p>
  
Tags: <deep-learning><deep-neural-networks><computer-vision><object-recognition>
Title: Precise localization and characterization of rudimentary shapes with neural networks
UserDisplayName: mbaytas
ThreadText: # Precise localization and characterization of rudimentary shapes with neural networks
  
  (Asked by: username_0 on 2017-05-10)
  
  I understand that there are flavors of (convolutional) neural networks that are useful for object [localization and detection](https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/) tasks of reasonable difficulty. In all of the examples I have seen so far, localization is formulated as finding the corners of a bounding box. Often, the fit is not expected to be very precise:
  
  [![traffic sign bounding box](https://i.sstatic.net/xmWck.jpg)](https://i.sstatic.net/xmWck.jpg)
  
  Conversely, I am interested in a task I want to achieve very precise localization and characterization of some simple shapes or objects. As an example of one of the simplest cases I can think of, my inputs will be images like the following:
  
  [![28 21 15](https://i.sstatic.net/NOqN5.jpg)](https://i.sstatic.net/NOqN5.jpg)
  
  Given this 60x60 image, I want my neural net(s), via **regression**, to tell me that the circle's **diameter is 18px** and its **centre is located at (28, 21)** from top left. (I will train it using similar 60x60 images with white circles of various sizes on black backgrounds.)
  
  Later I am interested in dealing with similar tasks in the real world, e.g. spheres/cubes/cylinders with different viewing angles, light conditions, occlusions, etc. However, I am interested in solving this simplest case first. (One reason is that I can generate this data very easily.)
  
  I have the following specific questions:
  
  1. Has anyone used neural nets for this sort of tasks before? (e.g. precisely determining sizes and centroids of objects)
  2. My understanding is that these things are at least theoretically possible using convolutional nets, or even sufficiently complicated vanilla fully connected nets. Is this correct?
  3. What architecture(s) would be appropriate for these tasks?
  
  Note: I am aware that fitting a bounding box to the circles and calculating its centre and size will solve this particular case, but it will not generalize to handle occlusions, changing lights, etc. I would like to move towards a method which can, for example, calculate the centroids and diameters of spheres in real-world B&W photos.
  
  **Tags:** deep-learning, deep-neural-networks, computer-vision, object-recognition
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-05-22. Score: 2
  
  I think you're describing "object localization and detection" which combines object identification with discovery of its spatial placement in the field of view. There's been a lot of work on this in the past 5 years using CNNs and variants. It's very much in demand for pedestrian/obstruction detection, identification, and avoidance in autonomous car systems.
  
  Here's a place to start: <https://github.com/kjw0612/awesome-deep-vision>
  
  Especially, take a look at "Faster R-CNN" by Ren et al, 2016.
  
  
  ---
  
  