Answers.Body: <p>from what I understand you are building you own model for this specific use case. From my perspective I would try <em>not to reinvent the wheel</em>, as it is said, and use an already proven and working model such as the YOLOs (<a href="https://arxiv.org/abs/1506.02640" rel="nofollow noreferrer">v1</a>, <a href="https://arxiv.org/abs/1612.08242" rel="nofollow noreferrer">v2</a> and <a href="https://arxiv.org/abs/1804.02767" rel="nofollow noreferrer">v3</a>).</p>
  
  <p>YOLO does not tell you the center pixel of the image directly but it tells you the center cell, with respect to the predicted object, of a grid (which is built on top of the image) responsible for predicting each object. See on the left image how the grid is built on top of the input image, then YOLO computes a probability map, and then the bounding boxes are predicted from the <strong>cell in the object's center</strong>. I have highlighted which cells would be responsible for predicting each object in the rightmost image. (Image from YOLOv1 paper)</p>
  
  <p><a href="https://i.sstatic.net/9DbkU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/9DbkU.png" alt="YOLO cell approach"></a></p>
  
  <p>The grid can have different resolutions but if you make it equal to the image size, then you would have the center pixel of each object predicted. This is because YOLO predicts objects in each cell of the grid, <strong><em>so if the grid is equal to the image size, YOLO will predict objects in each pixel</em></strong>.  </p>
  
  <p>As an example, imagine you have an input image of <span class="math-container">$[H \times W] = [416 \times 416]$</span> then YOLO would compute a grid of <span class="math-container">$[S_1 \times S_2]=[52 \times 52]$</span> on top of it. And predict objects in the center cell of the <span class="math-container">$[S_1 \times S_2]$</span>. So, if you tune YOLO for computing a grid such as <span class="math-container">$[S_1 \times S_2] = [H \times W]$</span>, then YOLO would output objects prediction with respect to the image pixels, in other words, YOLO would predict bounding boxes centered in the image pixel on the object's center.</p>
  
  <p>This is how I would proceed for this use case, I hope it helps you or at least give you some clues about how to proceed further! Cheers! :)</p>
  
  <hr>
  
  <p>NOTE: I chose the image size and grid size with numbers I usually see at work. Specifically, using YOLOv3. In YOLOv3, for aninput image of <span class="math-container">$[H \times W] = [416 \times 416]$</span>, 3 grids are built, with different resolutions (for predicting big and small objects), with the following grids sizes are: <span class="math-container">$[13 \times 13], [26 \times 26], [52 \times 52]$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-11T02:20:41.473Z
Answers.Text: Thanks for your reply. Is there any faster way to do this than with YOLO that you might know of? My dataset is very simplistic and could almost be solved by using a heuristic approach, were it not for minor differences in dimensions and proportions.
Answers.UserDisplayName: Recessive
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-11T10:26:59.783Z
Answers.Text: Yes, by building a custom model as you suggest, but you did not provide enough information about 2 key things: how you do the bounding box regression and what your loss function is (very important for custom models). By predicting the bbox coordinates (x,y,w,h) you can get the (x,y) pixel in the image. But before doing that you need to write an adequate loss function that facilitates that prediction.
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-07T09:08:33.570Z
Answers.LastActivityDate: 2020-02-07T09:08:33.570Z
Answers.UserDisplayName: JVGD
Answers.Body: <p>You must increase the percentage of &quot;1&quot; labels on the target output, so that &quot;all zeroes&quot; doesn't give such a low loss. If your images are <span class="math-container">$512 \times 512$</span>, could you label the target image as ones within a radius of 10 pixels? This way each target would have about 314 &quot;1&quot; labels (0.12%), rather than just one (0.0004%). Maybe even a radius of 20 pixels would be better.</p>
  <p>Also binary crossentropy is a better loss function for this than MSE (L2 difference).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-02-26T14:09:12.147Z
Answers.LastActivityDate: 2022-02-26T14:09:12.147Z
Answers.UserDisplayName: NikoNyrh
Body: <p>I want to make a model that outputs the centre pixel of objects appearing in an image.</p>
  
  <p>My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where each pixel has a value of 1 if it is the center of an object and 0 otherwise. Each input image has roughly ~80 objects.</p>
  
  <p>The problem with this is the CNN learns the easiest way to reduce the error, which is having the entire output be 0, because for 97% of cases that's correct. As such, error decreases but it learns nothing.</p>
  
  <p>What is another potential method for training a network to do something similar? I also tried adding dropout, which made the output a lot more noisy and it seemed to learn ok, but eventually ended up in the same state as before with the entire output being 0, never really seeming to learn how to output the locations of objects.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-07T10:57:13.057Z
Comments.Text: I didn't saw your dataset but problem can also occur when one time pixel RGB(10, 10, 10) is 1, another time is 0
Comments.UserDisplayName: Adrian Grygutis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-12T15:49:03.313Z
Comments.Text: Do you have to train a new model or can you use a pre-trained model - it might be enough to remove the last one or two layers and adjust those to your needs? For object detection there are basically two approaches: Instance Segmentation that try to find a pixel-by-pixel classification and Bounding Box models that find the minimum box to contain the entire object. A _centre pixel_ of an object can have different meaning for these approaches: is the centre pixel of the crescent the void in the middle of the bounding box?
Comments.UserDisplayName: Gregor
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-12T23:09:52.043Z
Comments.Text: @Gregor Yes the centre pixel would be the middle of the void in the bounding box. I could use a pre-trained model, but most of those models feel much to large for the task at hand
Comments.UserDisplayName: Recessive
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-02-13T12:18:16.273Z
Comments.Text: A pre-trained model is indeed probably fairly large for your purpose (usually trained to predict 1000 classes) but they come almost production-ready: set up a machine with TensorFlow, OpenCV, ... and load the model. That's it. The outputs are the probabilities of the different classes (this is where you might want to adjust your final layer to predict only those 80 classes) and the bounding box or pixel mask. The centre of a bounding box is simply the middle of the height and width.
  
  You can also find a trade-off between accuracy and speed by adjusting the precision from FP32 to maybe FP16.
Comments.UserDisplayName: Gregor
Tags: <convolutional-neural-networks><object-detection><regression>
Title: Possible model to use to find pixel locations of objects
UserDisplayName: Recessive
ThreadText: # Possible model to use to find pixel locations of objects
  
  (Asked by: username_0 on 2020-02-03)
  
  I want to make a model that outputs the centre pixel of objects appearing in an image.
  
  My current method involves using a CNN with L2 loss to output an image of equivalent size to the input where each pixel has a value of 1 if it is the center of an object and 0 otherwise. Each input image has roughly ~80 objects.
  
  The problem with this is the CNN learns the easiest way to reduce the error, which is having the entire output be 0, because for 97% of cases that's correct. As such, error decreases but it learns nothing.
  
  What is another potential method for training a network to do something similar? I also tried adding dropout, which made the output a lot more noisy and it seemed to learn ok, but eventually ended up in the same state as before with the entire output being 0, never really seeming to learn how to output the locations of objects.
  
  **Tags:** convolutional-neural-networks, object-detection, regression
  > **Comments:**
  > **username_1**: I didn't saw your dataset but problem can also occur when one time pixel RGB(10, 10, 10) is 1, another time is 0  
  > **username_2**: Do you have to train a new model or can you use a pre-trained model - it might be enough to remove the last one or two layers and adjust those to your needs? For object detection there are basically two approaches: Instance Segmentation that try to find a pixel-by-pixel classification and Bounding Box models that find the minimum box to contain the entire object. A _centre pixel_ of an object can have different meaning for these approaches: is the centre pixel of the crescent the void in the middle of the bounding box?  
  > **username_0**: @Gregor Yes the centre pixel would be the middle of the void in the bounding box. I could use a pre-trained model, but most of those models feel much to large for the task at hand  
  > **username_2**: A pre-trained model is indeed probably fairly large for your purpose (usually trained to predict 1000 classes) but they come almost production-ready: set up a machine with TensorFlow, OpenCV, ... and load the model. That's it. The outputs are the probabilities of the different classes (this is where you might want to adjust your final layer to predict only those 80 classes) and the bounding box or pixel mask. The centre of a bounding box is simply the middle of the height and width.  You can also find a trade-off between accuracy and speed by adjusting the precision from FP32 to maybe FP16.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2020-02-07. Score: 1
  
  from what I understand you are building you own model for this specific use case. From my perspective I would try *not to reinvent the wheel*, as it is said, and use an already proven and working model such as the YOLOs ([v1](https://arxiv.org/abs/1506.02640), [v2](https://arxiv.org/abs/1612.08242) and [v3](https://arxiv.org/abs/1804.02767)).
  
  YOLO does not tell you the center pixel of the image directly but it tells you the center cell, with respect to the predicted object, of a grid (which is built on top of the image) responsible for predicting each object. See on the left image how the grid is built on top of the input image, then YOLO computes a probability map, and then the bounding boxes are predicted from the **cell in the object's center**. I have highlighted which cells would be responsible for predicting each object in the rightmost image. (Image from YOLOv1 paper)
  
  [![YOLO cell approach](https://i.sstatic.net/9DbkU.png)](https://i.sstatic.net/9DbkU.png)
  
  The grid can have different resolutions but if you make it equal to the image size, then you would have the center pixel of each object predicted. This is because YOLO predicts objects in each cell of the grid, ***so if the grid is equal to the image size, YOLO will predict objects in each pixel***.
  
  As an example, imagine you have an input image of $[H \times W] = [416 \times 416]$ then YOLO would compute a grid of $[S\_1 \times S\_2]=[52 \times 52]$ on top of it. And predict objects in the center cell of the $[S\_1 \times S\_2]$. So, if you tune YOLO for computing a grid such as $[S\_1 \times S\_2] = [H \times W]$, then YOLO would output objects prediction with respect to the image pixels, in other words, YOLO would predict bounding boxes centered in the image pixel on the object's center.
  
  This is how I would proceed for this use case, I hope it helps you or at least give you some clues about how to proceed further! Cheers! :)
  
  ---
  
  NOTE: I chose the image size and grid size with numbers I usually see at work. Specifically, using YOLOv3. In YOLOv3, for aninput image of $[H \times W] = [416 \times 416]$, 3 grids are built, with different resolutions (for predicting big and small objects), with the following grids sizes are: $[13 \times 13], [26 \times 26], [52 \times 52]$
  
  > **Comments:**
  > **username_0**: Thanks for your reply. Is there any faster way to do this than with YOLO that you might know of? My dataset is very simplistic and could almost be solved by using a heuristic approach, were it not for minor differences in dimensions and proportions.  
  > **username_3**: Yes, by building a custom model as you suggest, but you did not provide enough information about 2 key things: how you do the bounding box regression and what your loss function is (very important for custom models). By predicting the bbox coordinates (x,y,w,h) you can get the (x,y) pixel in the image. But before doing that you need to write an adequate loss function that facilitates that prediction.  
  
  ---
  
  ### Answer by username_4 on 2022-02-26. Score: 0
  
  You must increase the percentage of "1" labels on the target output, so that "all zeroes" doesn't give such a low loss. If your images are $512 \times 512$, could you label the target image as ones within a radius of 10 pixels? This way each target would have about 314 "1" labels (0.12%), rather than just one (0.0004%). Maybe even a radius of 20 pixels would be better.
  
  Also binary crossentropy is a better loss function for this than MSE (L2 difference).
  
  
  ---
  
  