Answers.Body: <p><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6112046/#sec3-sensors-18-02702title" rel="nofollow noreferrer">Here's</a> a recent paper that does what you're looking for. It looks like they achieve this simply by adding a couple rotated prior boxes and regressing the angles in between.  This is similar to what standard object detectors do in terms of creating a bunch of prior box shapes and regressing the actual sizes.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-11T22:57:48.117Z
Answers.LastActivityDate: 2019-01-11T22:57:48.117Z
Answers.UserDisplayName: juicedatom
Answers.Body: <p>It should not be much more difficult to predict a rotated rectangle compared to a bounding box.</p>
  <p>A bounding box can be parameterized with 4 floats: <span class="math-container">$x_c$</span>, <span class="math-container">$y_c$</span>, width, height.</p>
  <p>A rotated rectangle can be parameterized with 5 floats: <span class="math-container">$x_c$</span>, <span class="math-container">$y_c$</span>, width, height, angle.</p>
  <p>However, to avoid the wrap-around issue with predicting the angle with one value (0° is same as 360°), <strong>it should be better to predict sine and cosine instead</strong>.</p>
  <p>It is actually useful to predict rotated rectangles for text detection (each text field is a rotated rectangle).  Indeed, in the wild, text can be in any orientation, and it is important to predict precisely rotated rectangles for OCR to work well. It is especially true for long text boxes near 45° (an axis-aligned bounding box around this would be useless because too big).</p>
  <p>Here are 2 links I found about this topic:</p>
  <ul>
  <li><p><a href="https://github.com/ultralytics/yolov3/issues/345" rel="nofollow noreferrer">Rotated Bounding boxes training</a></p>
  </li>
  <li><p><a href="https://arxiv.org/abs/1703.01086" rel="nofollow noreferrer">Arbitrary-Oriented Scene Text Detection via Rotation Proposals</a></p>
  </li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-12T17:17:31.120Z
Answers.LastActivityDate: 2021-01-28T23:41:35.277Z
Answers.LastEditDate: 2021-01-28T23:41:35.277Z
Answers.UserDisplayName: Ismael EL ATIFI
Answers.Body: <p><strong>Cartesian Bias and Pipeline Efficiency</strong></p>
  
  <p>You are experiencing a techno-cultural artifact of Cartesian-centric imaging running all the way back to the dawn of coordinate systems. It is the momentum of practice as a consequence of applying Cartesian 2D coordinates to rasterize images appearing at the focal planes of lenses from the dawn of television and the earliest standards of raster based capture and display.</p>
  
  <p>Although some work was done toward adding tilt to bounding rectangles in the late 1990s and since, from a time and computing resource conservation perspective, it is computationally and programmatically less costly to include the four useless triangles of pixels and keep the bounding box orthogonal with the pixel grid.</p>
  
  <p>Adding a tilt angle to the bounding boxes is marginally competitive when detecting ships from a satellite only because two conditions offset the inefficiencies in that narrow domain. The ship appears as an oblong rectangle with rounded corners from a satellite positioned in geosynchronous orbit. In the general case, adding a tilt angle can slow recognition significantly.</p>
  
  <p><strong>Biology Less Biased</strong></p>
  
  <p>An interesting side note is that the neural networks of animal and human vision systems do not have that Cartesian-centricity, but that doesn't help this question's solution, since non-orthogonal hardware and software is virtually nonexistent.</p>
  
  <p><strong>Early Non-Cartesian Research and Today's Rasterization</strong></p>
  
  <p>Gerber Scientific Techonology research and development in the 1980s (South Windsor, Connecticut, U.S.) investigated vector capture, storage, and display, but the R&amp;D was not financially sustainable for a mid-side technology corporation for the reasons above.</p>
  
  <p>What remains, because it is economically viable and necessary from an animation point of view, is rasterization on the end of the system that converts vector models into frames of pixels. We see this in on the rendering  SVG, VRML, and the original intent of CUDA cores and other hardware rendering acceleration strategies and architectures.</p>
  
  <p>On the object and action recognition side, the support of vector models directly from imaging is much less developed. This has not been a major stumbling block for computer vision because the wasted pixels at one tilt angle may be of central importance at another tilt angle, so there are no actual wasted input pixels if the centering of key scene elements is widely distributed in translation and tilt, which is often the case in real life (although not so much in hygienically pre-processed datasets).</p>
  
  <p><strong>Conventions Around Object Minus Camera Tilt and Skew from Parallax</strong></p>
  
  <p>Once edge detection, interior-versus-exterior, and 3D solid recognition come into play, the design of CNN pipelines and the way kernels can do radial transformation without actually requiring <span class="math-container">$\; \sin, \, \cos, \, \text{and} \, \arctan \;$</span> functions evaporate the computational burden of the Cartesian nature of pixel tensors. The end result is that the bounding box being orthogonal to the image frame is not as problematic as it initially appears. Efforts to conserve the four triangles of pixels and pre-process orientation is often a wasted effort by a gross margin.</p>
  
  <p><strong>Summary</strong></p>
  
  <p>The bottom line is that efforts to produce vector recognition from roster inputs have been significantly inferior in terms of resource and wait time burden, with the exception of insignificant gains in the narrow domain of naval reconnaissance satellite images. Trigonometry is expensive, but convolution kernels, especially now that they are moving from software into hardware accelerated computing paths in VLSI, is computable at lower costs.</p>
  
  <p><strong>Past and Current Work</strong></p>
  
  <p>Below is some work that deals with tilting with regard to objects and the effects of parallax in relation to the Cartesian coordinate system of the raster representation. Most of the work has to do with recognizing 3D objects in a 3D coordinate system to project trajectories and pilot or drive vehicles rationally on the basis of Newtonian mechanics.</p>
  
  <p><a href="ftp://crack.seismo.unr.edu/downloads/russell/trees/klosowski_1998_efficient_collision_k-DOPs.PDF" rel="nofollow noreferrer"><em>Efficient Collision Detection Using Bounding Volume Hierarchies of k-DOPs</em></a>, James T. Klosowski, Martin Held, Joseph S.B. Mitchell, Henry Sowizral, and Karel Zikan, 1998</p>
  
  <p><a href="https://pdfs.semanticscholar.org/832e/eaede4f697f005b007798fe0d04829f656b7.pdf" rel="nofollow noreferrer"><em>Sliding Shapes for 3D Object Detection in Depth Images</em></a>, Shuran Song and Jianxiong Xiao, 2014</p>
  
  <p><a href="https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kar_Amodal_Completion_and_ICCV_2015_paper.pdf" rel="nofollow noreferrer"><em>Amodal Completion and Size Constancy in Natural Scenes</em></a>, Abhishek Kar, Shubham Tulsiani, Joao Carreira and Jitendra Malik, 2015</p>
  
  <p><a href="https://arxiv.org/pdf/1609.04147.pdf" rel="nofollow noreferrer"><em>HMD Vision-based Teleoperating UGV and UAV for Hostile
  Environment using Deep Learning</em></a>, Abhishek Sawarkar1, Vishal Chaudhari, Rahul Chavan, Varun Zope, Akshay Budale and Faruk Kazi, 2016</p>
  
  <p><a href="http://159.226.21.68/bitstream/173211/14544/1/Ship%20Rotated%20Bounding%20Box%20Space%20for%20Ship%20Extraction%20from%20High-Resolution%20Optical%20Satellite%20Images%20with%20Complex%20Backgrounds.pdf" rel="nofollow noreferrer"><em>Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds</em></a>, Z Liu, H Wang, L Weng, Y Yang, 2016</p>
  
  <p><a href="http://www.dnnseye.com/publications/papers/zhuo_3det_cvpr17.pdf" rel="nofollow noreferrer"><em>Amodal Detection of 3D Objects:
  Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Images</em></a>, Zhuo Deng, 2017</p>
  
  <p><a href="http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w31/Mahendran_3D_Pose_Regression_ICCV_2017_paper.pdf0" rel="nofollow noreferrer"><em>3D Pose Regression using Convolutional Neural Networks</em></a>, Siddharth Mahendran, 2017</p>
  
  <p><a href="https://www.mdpi.com/2226-4310/4/2/32/htm" rel="nofollow noreferrer"><em>Aerial Target Tracking Algorithm Based on Faster R-CNN Combined with Frame Differencing</em></a>, Yurong Yang, Huajun Gong, Xinhua Wang and Peng Sun, 2017</p>
  
  <p><a href="http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w3/Kumar_A_Semi-Automatic_2D_CVPR_2018_paper.pdf" rel="nofollow noreferrer"><em>A Semi-Automatic 2D solution for Vehicle Speed Estimation from Monocular Videos</em></a>, Amit Kumar, Pirazh Khorramshahi, Wei-An Lin, Prithviraj Dhar, Jun-Cheng Chen, Rama Chellappa, 2018</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-01-15T05:58:46.263Z
Answers.LastActivityDate: 2019-01-16T09:10:06.290Z
Answers.LastEditDate: 2019-01-16T09:10:06.290Z
Answers.UserDisplayName: Douglas Daseeco
Body: <p>I have checked out many methods and papers, like YOLO, SSD, etc., with good results in detecting a rectangular box around an object, However, I could not find any paper that shows a method that learns a rotated bounding box.</p>
  <p>Is it difficult to learn the rotated bounding box for a (rotated) object?</p>
  <p>Here's a diagram that illustrates the problem.</p>
  <p><a href="https://i.sstatic.net/Ljimz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Ljimz.png" alt="" /></a></p>
  <p>For example, for this object (see <a href="https://stackoverflow.com/q/40404031/9277245">this</a>), its bounding box should be of the same shape (the rotated rectangle is shown in the 2nd right image), but the prediction result for the YOLO will be Ist right.</p>
  <p>Is there any research paper that tackles this problem?</p>
  
Tags: <convolutional-neural-networks><computer-vision><object-detection><yolo>
Title: Is it difficult to learn the rotated bounding box for a (rotated) object?
UserDisplayName: Ankish Bansal
ThreadText: # Is it difficult to learn the rotated bounding box for a (rotated) object?
  
  (Asked by: username_0 on 2019-01-11)
  
  I have checked out many methods and papers, like YOLO, SSD, etc., with good results in detecting a rectangular box around an object, However, I could not find any paper that shows a method that learns a rotated bounding box.
  
  Is it difficult to learn the rotated bounding box for a (rotated) object?
  
  Here's a diagram that illustrates the problem.
  
  [![](https://i.sstatic.net/Ljimz.png)](https://i.sstatic.net/Ljimz.png)
  
  For example, for this object (see [this](https://stackoverflow.com/q/40404031/9277245)), its bounding box should be of the same shape (the rotated rectangle is shown in the 2nd right image), but the prediction result for the YOLO will be Ist right.
  
  Is there any research paper that tackles this problem?
  
  **Tags:** convolutional-neural-networks, computer-vision, object-detection, yolo
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2019-01-15. Score: 3 | Accepted answer
  
  **Cartesian Bias and Pipeline Efficiency**
  
  You are experiencing a techno-cultural artifact of Cartesian-centric imaging running all the way back to the dawn of coordinate systems. It is the momentum of practice as a consequence of applying Cartesian 2D coordinates to rasterize images appearing at the focal planes of lenses from the dawn of television and the earliest standards of raster based capture and display.
  
  Although some work was done toward adding tilt to bounding rectangles in the late 1990s and since, from a time and computing resource conservation perspective, it is computationally and programmatically less costly to include the four useless triangles of pixels and keep the bounding box orthogonal with the pixel grid.
  
  Adding a tilt angle to the bounding boxes is marginally competitive when detecting ships from a satellite only because two conditions offset the inefficiencies in that narrow domain. The ship appears as an oblong rectangle with rounded corners from a satellite positioned in geosynchronous orbit. In the general case, adding a tilt angle can slow recognition significantly.
  
  **Biology Less Biased**
  
  An interesting side note is that the neural networks of animal and human vision systems do not have that Cartesian-centricity, but that doesn't help this question's solution, since non-orthogonal hardware and software is virtually nonexistent.
  
  **Early Non-Cartesian Research and Today's Rasterization**
  
  Gerber Scientific Techonology research and development in the 1980s (South Windsor, Connecticut, U.S.) investigated vector capture, storage, and display, but the R&D was not financially sustainable for a mid-side technology corporation for the reasons above.
  
  What remains, because it is economically viable and necessary from an animation point of view, is rasterization on the end of the system that converts vector models into frames of pixels. We see this in on the rendering SVG, VRML, and the original intent of CUDA cores and other hardware rendering acceleration strategies and architectures.
  
  On the object and action recognition side, the support of vector models directly from imaging is much less developed. This has not been a major stumbling block for computer vision because the wasted pixels at one tilt angle may be of central importance at another tilt angle, so there are no actual wasted input pixels if the centering of key scene elements is widely distributed in translation and tilt, which is often the case in real life (although not so much in hygienically pre-processed datasets).
  
  **Conventions Around Object Minus Camera Tilt and Skew from Parallax**
  
  Once edge detection, interior-versus-exterior, and 3D solid recognition come into play, the design of CNN pipelines and the way kernels can do radial transformation without actually requiring $\; \sin, \, \cos, \, \text{and} \, \arctan \;$ functions evaporate the computational burden of the Cartesian nature of pixel tensors. The end result is that the bounding box being orthogonal to the image frame is not as problematic as it initially appears. Efforts to conserve the four triangles of pixels and pre-process orientation is often a wasted effort by a gross margin.
  
  **Summary**
  
  The bottom line is that efforts to produce vector recognition from roster inputs have been significantly inferior in terms of resource and wait time burden, with the exception of insignificant gains in the narrow domain of naval reconnaissance satellite images. Trigonometry is expensive, but convolution kernels, especially now that they are moving from software into hardware accelerated computing paths in VLSI, is computable at lower costs.
  
  **Past and Current Work**
  
  Below is some work that deals with tilting with regard to objects and the effects of parallax in relation to the Cartesian coordinate system of the raster representation. Most of the work has to do with recognizing 3D objects in a 3D coordinate system to project trajectories and pilot or drive vehicles rationally on the basis of Newtonian mechanics.
  
  [*Efficient Collision Detection Using Bounding Volume Hierarchies of k-DOPs*](ftp://crack.seismo.unr.edu/downloads/russell/trees/klosowski_1998_efficient_collision_k-DOPs.PDF), James T. Klosowski, Martin Held, Joseph S.B. Mitchell, Henry Sowizral, and Karel Zikan, 1998
  
  [*Sliding Shapes for 3D Object Detection in Depth Images*](https://pdfs.semanticscholar.org/832e/eaede4f697f005b007798fe0d04829f656b7.pdf), Shuran Song and Jianxiong Xiao, 2014
  
  [*Amodal Completion and Size Constancy in Natural Scenes*](https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kar_Amodal_Completion_and_ICCV_2015_paper.pdf), Abhishek Kar, Shubham Tulsiani, Joao Carreira and Jitendra Malik, 2015
  
  [*HMD Vision-based Teleoperating UGV and UAV for Hostile
  Environment using Deep Learning*](https://arxiv.org/pdf/1609.04147.pdf), Abhishek Sawarkar1, Vishal Chaudhari, Rahul Chavan, Varun Zope, Akshay Budale and Faruk Kazi, 2016
  
  [*Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds*](http://159.226.21.68/bitstream/173211/14544/1/Ship%20Rotated%20Bounding%20Box%20Space%20for%20Ship%20Extraction%20from%20High-Resolution%20Optical%20Satellite%20Images%20with%20Complex%20Backgrounds.pdf), Z Liu, H Wang, L Weng, Y Yang, 2016
  
  [*Amodal Detection of 3D Objects:
  Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Images*](http://www.dnnseye.com/publications/papers/zhuo_3det_cvpr17.pdf), Zhuo Deng, 2017
  
  [*3D Pose Regression using Convolutional Neural Networks*](http://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w31/Mahendran_3D_Pose_Regression_ICCV_2017_paper.pdf0), Siddharth Mahendran, 2017
  
  [*Aerial Target Tracking Algorithm Based on Faster R-CNN Combined with Frame Differencing*](https://www.mdpi.com/2226-4310/4/2/32/htm), Yurong Yang, Huajun Gong, Xinhua Wang and Peng Sun, 2017
  
  [*A Semi-Automatic 2D solution for Vehicle Speed Estimation from Monocular Videos*](http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w3/Kumar_A_Semi-Automatic_2D_CVPR_2018_paper.pdf), Amit Kumar, Pirazh Khorramshahi, Wei-An Lin, Prithviraj Dhar, Jun-Cheng Chen, Rama Chellappa, 2018
  
  
  ---
  
  ### Answer by username_1 on 2019-01-11. Score: 3
  
  [Here's](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6112046/#sec3-sensors-18-02702title) a recent paper that does what you're looking for. It looks like they achieve this simply by adding a couple rotated prior boxes and regressing the angles in between. This is similar to what standard object detectors do in terms of creating a bunch of prior box shapes and regressing the actual sizes.
  
  
  ---
  
  ### Answer by username_2 on 2020-06-12. Score: 2
  
  It should not be much more difficult to predict a rotated rectangle compared to a bounding box.
  
  A bounding box can be parameterized with 4 floats: $x\_c$, $y\_c$, width, height.
  
  A rotated rectangle can be parameterized with 5 floats: $x\_c$, $y\_c$, width, height, angle.
  
  However, to avoid the wrap-around issue with predicting the angle with one value (0° is same as 360°), **it should be better to predict sine and cosine instead**.
  
  It is actually useful to predict rotated rectangles for text detection (each text field is a rotated rectangle). Indeed, in the wild, text can be in any orientation, and it is important to predict precisely rotated rectangles for OCR to work well. It is especially true for long text boxes near 45° (an axis-aligned bounding box around this would be useless because too big).
  
  Here are 2 links I found about this topic:
  
  * [Rotated Bounding boxes training](https://github.com/ultralytics/yolov3/issues/345)
  * [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://arxiv.org/abs/1703.01086)
  
  
  ---
  
  