Body: <p>Currently, I am looking at how Mask R-CNN works. It has a backbone, RPN, heads, etc. The backbone is used for creating the feature maps, which are then passed to the RPN to create proposals. Those proposals would then be aligned with feature maps and rescaled to some <span class="math-container">$n \times n$</span> pixels before entering box head or mask head or keypoint head.</p>
  <p>Since conv2D is not scale-invariant, I think this scaling to <span class="math-container">$n \times n$</span>  would introduce scale-invariant characteristics.</p>
  <p>For an object that is occluded or truncated, I think scaling to <span class="math-container">$n \times n$</span> is not really appropriate.</p>
  <p>Is it possible if I predict the visibility of the object inside the box head (outputting not only xyxy [bounding box output], but also xyxy+x_size y_size [bounding box output  + width height scale of object]). This x_size and y_size value would then be used to rescale <span class="math-container">$n \times n$</span>  input.</p>
  <p>So, if only half of the object is seen (occluded or truncated), inputs inside the keypoint head or mask head would be 0.5x by 0.5x.</p>
  <p>Is this a good approach to counter occlusion and truncation?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-07T02:16:59.413Z
Comments.Text: Can you please clarify this part "outputting not only xyxy, but also xyxy x_size y_size"? What `xyxy` and `xyxy x_size y_size`? What does that mean? Note that I changed from x by x to $n \times n$ because of latex.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-08T00:39:48.350Z
Comments.Text: Editted for clarifitcation xyxy means xmin, ymin, xmax, ymax (i believe this is bounding box output) or xywh (xmin, ymin, width, height)
Comments.UserDisplayName: Darwin Harianto
Tags: <convolutional-neural-networks><object-detection><mask-rcnn>
Title: Improving Mask RCNN by arbitrary scaling head input
UserDisplayName: Darwin Harianto
ThreadText: # Improving Mask RCNN by arbitrary scaling head input
  
  (Asked by: username_0 on 2021-02-03)
  
  Currently, I am looking at how Mask R-CNN works. It has a backbone, RPN, heads, etc. The backbone is used for creating the feature maps, which are then passed to the RPN to create proposals. Those proposals would then be aligned with feature maps and rescaled to some $n \times n$ pixels before entering box head or mask head or keypoint head.
  
  Since conv2D is not scale-invariant, I think this scaling to $n \times n$ would introduce scale-invariant characteristics.
  
  For an object that is occluded or truncated, I think scaling to $n \times n$ is not really appropriate.
  
  Is it possible if I predict the visibility of the object inside the box head (outputting not only xyxy [bounding box output], but also xyxy+x\_size y\_size [bounding box output + width height scale of object]). This x\_size and y\_size value would then be used to rescale $n \times n$ input.
  
  So, if only half of the object is seen (occluded or truncated), inputs inside the keypoint head or mask head would be 0.5x by 0.5x.
  
  Is this a good approach to counter occlusion and truncation?
  
  **Tags:** convolutional-neural-networks, object-detection, mask-rcnn
  > **Comments:**
  > **username_1**: Can you please clarify this part "outputting not only xyxy, but also xyxy x_size y_size"? What `xyxy` and `xyxy x_size y_size`? What does that mean? Note that I changed from x by x to $n \times n$ because of latex.  
  > **username_0**: Editted for clarifitcation xyxy means xmin, ymin, xmax, ymax (i believe this is bounding box output) or xywh (xmin, ymin, width, height)  
  
  ---
  
  ## 0 answers
  
  