Answers.Body: <p>Heatmap in the sense of Corner net is the heatmap of the pooled corner values. As discussed in the paper, there is a corner pooling operation that gives you the vector values for a pixel point being a corner(it may or not). The output from the corner pooling is CxHxW. Then to generate a heat map you train the network similar to the Grad-CAM method. Training for heatmap generation means that we are going to tell how much a given pixel point has a weight to be a corner point. i.e if it has a high weight it will generate more colored values on that pixel area.</p>
  <p>The heatmap is used for the prediction of corner points. That means if you know which pixel has a higher weight then that pixel will have a higher probability to hold the top-right or bottom-left corner of the given bounding box.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-05-06T11:22:10.073Z
Answers.LastActivityDate: 2021-05-06T11:22:10.073Z
Answers.UserDisplayName: fetulhak abdurahman
Body: <p>I have been working on understanding how <a href="https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf" rel="nofollow noreferrer">CornerNet</a> works, but I couldn't figure out a few parts about the architecture.</p>
  <p>First, the authors mention that there are 3 distinct parts to be predicted as a heatmap, embedding, and offset.</p>
  <p>Also, in <a href="https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf" rel="nofollow noreferrer">the paper</a>, it is stated that the network was trained on the COCO dataset, which has bounding box and class annotations.</p>
  <p>As far as I am concerned, since CornerNet is based on detecting the top-left and bottom-right corners, the ground-truth labels for heatmap should be composed of top-left and bottom-right pixel locations of bounding boxes with the class score (but I might be wrong). What is the heatmap used for?</p>
  <p>Moreover, for the embedding part, authors used the pull&amp;push loss at the ground-truth pixel locations to find out which corner pairs belong to which object, but I don't understand how to backpropagate this loss. How do I back-propagate the embedding loss?</p>
  
Tags: <deep-learning><computer-vision><object-detection><papers>
Title: What is a heatmap in the CornerNet paper?
UserDisplayName: efedoganay
ThreadText: # What is a heatmap in the CornerNet paper?
  
  (Asked by: username_0 on 2020-07-10)
  
  I have been working on understanding how [CornerNet](https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf) works, but I couldn't figure out a few parts about the architecture.
  
  First, the authors mention that there are 3 distinct parts to be predicted as a heatmap, embedding, and offset.
  
  Also, in [the paper](https://openaccess.thecvf.com/content_ECCV_2018/papers/Hei_Law_CornerNet_Detecting_Objects_ECCV_2018_paper.pdf), it is stated that the network was trained on the COCO dataset, which has bounding box and class annotations.
  
  As far as I am concerned, since CornerNet is based on detecting the top-left and bottom-right corners, the ground-truth labels for heatmap should be composed of top-left and bottom-right pixel locations of bounding boxes with the class score (but I might be wrong). What is the heatmap used for?
  
  Moreover, for the embedding part, authors used the pull&push loss at the ground-truth pixel locations to find out which corner pairs belong to which object, but I don't understand how to backpropagate this loss. How do I back-propagate the embedding loss?
  
  **Tags:** deep-learning, computer-vision, object-detection, papers
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-05-06. Score: 1
  
  Heatmap in the sense of Corner net is the heatmap of the pooled corner values. As discussed in the paper, there is a corner pooling operation that gives you the vector values for a pixel point being a corner(it may or not). The output from the corner pooling is CxHxW. Then to generate a heat map you train the network similar to the Grad-CAM method. Training for heatmap generation means that we are going to tell how much a given pixel point has a weight to be a corner point. i.e if it has a high weight it will generate more colored values on that pixel area.
  
  The heatmap is used for the prediction of corner points. That means if you know which pixel has a higher weight then that pixel will have a higher probability to hold the top-right or bottom-left corner of the given bounding box.
  
  
  ---
  
  