Answers.Body: <p>Yes, theoretically it is possible to learn the offsets to get any possible bounding box from only one anchor box. However, it is hard to learn such dramatic shifts and changes. Learning only small offsets from the prior is easier and tends to converge better.</p>
  <p>In specific applications however, one might already know the typical size and ratio of objects, and that this is very similar for all of them. In such cases, one box per anchor can be enough to learn well.</p>
  <p>Note that many redundant boxes are predicted anyway, even if only one anchor box is used per location, because there are usually many anchor locations distributed in a grid based fashion over the image. Therefore, NMS is a necessary step anyway and does not depend on having multiple boxes per anchor.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-04T13:07:52.207Z
Answers.LastActivityDate: 2021-10-04T13:07:52.207Z
Answers.UserDisplayName: a crazy Minion
Body: <p>In various neural network detection pipelines, the detection works as follows:</p>
  <ol>
  <li>One processes the input image through the pretrained backbone</li>
  <li>Some additional convolutional layers</li>
  <li>The detection head, where each pixel on the given feauture map predicts the following:
  <ul>
  <li>Offset from the center of the cell (<span class="math-container">$\sigma(t_x), \sigma(t_y)$</span> on the image)</li>
  <li>Height and width of the bounding boxes <span class="math-container">$b_w, b_h$</span></li>
  <li>Objectness scores (probability of object presence)</li>
  <li>Class probabilities</li>
  </ul>
  </li>
  </ol>
  <p><a href="https://i.sstatic.net/qT07Y.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qT07Y.png" alt="enter image description here" /></a></p>
  <p>Usually, detection heads produce not a single box, but multiple.</p>
  <ul>
  <li>The first version of <a href="https://arxiv.org/abs/1506.02640" rel="nofollow noreferrer">YOLO</a> - outputs 2 boxes per location on the feature map of size <span class="math-container">$7 \times 7$</span></li>
  <li><a href="https://arxiv.org/pdf/1506.01497.pdf" rel="nofollow noreferrer">Faster R-CNN</a> outputs 9 boxes per location</li>
  <li><a href="https://arxiv.org/abs/1804.02767" rel="nofollow noreferrer">YOLO v3</a>  - outputs 9 boxes per pixel from the predefined anchors : <code>(10×13),(16×30),(33×23),(30×61),(62×45),(59× 119), (116 × 90), (156 × 198), (373 × 326)</code></li>
  </ul>
  <p>These anchors give the priors for the bounding boxes, but with the help
  of <span class="math-container">$\sigma(t_x), \sigma(t_y), b_w, b_h$</span> one can get any possible bounding box on the image for some pixel on the feature map.</p>
  <p>Therefore, the network will produce plenty of redundant boxes, and a certain procedure - <a href="https://towardsdatascience.com/non-maximum-suppression-nms-93ce178e177c" rel="nofollow noreferrer">NMS suppresion</a> has to be run over the bounding box predictions to select only the best.</p>
  <p>Or the purpose of these anchors is to start from a prior, reshape and shift slightly the bounding box, and then compare with the ground truth.</p>
  <p>Is it the case, that if one used only a single bounding box for detection - it would be hard to train the network to rescale the initial bounding to, say, 10 times, and produce some specific aspect ratio?</p>
  
Tags: <computer-vision><object-detection><bounding-box>
Title: Why do the object detection networks produce multiple anchor boxes per location?
UserDisplayName: spiridon_the_sun_rotator
ThreadText: # Why do the object detection networks produce multiple anchor boxes per location?
  
  (Asked by: username_0 on 2021-10-02)
  
  In various neural network detection pipelines, the detection works as follows:
  
  1. One processes the input image through the pretrained backbone
  2. Some additional convolutional layers
  3. The detection head, where each pixel on the given feauture map predicts the following:
     * Offset from the center of the cell ($\sigma(t\_x), \sigma(t\_y)$ on the image)
     * Height and width of the bounding boxes $b\_w, b\_h$
     * Objectness scores (probability of object presence)
     * Class probabilities
  
  [![enter image description here](https://i.sstatic.net/qT07Y.png)](https://i.sstatic.net/qT07Y.png)
  
  Usually, detection heads produce not a single box, but multiple.
  
  * The first version of [YOLO](https://arxiv.org/abs/1506.02640) - outputs 2 boxes per location on the feature map of size $7 \times 7$
  * [Faster R-CNN](https://arxiv.org/pdf/1506.01497.pdf) outputs 9 boxes per location
  * [YOLO v3](https://arxiv.org/abs/1804.02767) - outputs 9 boxes per pixel from the predefined anchors : `(10×13),(16×30),(33×23),(30×61),(62×45),(59× 119), (116 × 90), (156 × 198), (373 × 326)`
  
  These anchors give the priors for the bounding boxes, but with the help
  of $\sigma(t\_x), \sigma(t\_y), b\_w, b\_h$ one can get any possible bounding box on the image for some pixel on the feature map.
  
  Therefore, the network will produce plenty of redundant boxes, and a certain procedure - [NMS suppresion](https://towardsdatascience.com/non-maximum-suppression-nms-93ce178e177c) has to be run over the bounding box predictions to select only the best.
  
  Or the purpose of these anchors is to start from a prior, reshape and shift slightly the bounding box, and then compare with the ground truth.
  
  Is it the case, that if one used only a single bounding box for detection - it would be hard to train the network to rescale the initial bounding to, say, 10 times, and produce some specific aspect ratio?
  
  **Tags:** computer-vision, object-detection, bounding-box
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-10-04. Score: 1 | Accepted answer
  
  Yes, theoretically it is possible to learn the offsets to get any possible bounding box from only one anchor box. However, it is hard to learn such dramatic shifts and changes. Learning only small offsets from the prior is easier and tends to converge better.
  
  In specific applications however, one might already know the typical size and ratio of objects, and that this is very similar for all of them. In such cases, one box per anchor can be enough to learn well.
  
  Note that many redundant boxes are predicted anyway, even if only one anchor box is used per location, because there are usually many anchor locations distributed in a grid based fashion over the image. Therefore, NMS is a necessary step anyway and does not depend on having multiple boxes per anchor.
  
  
  ---
  
  