title: Estimating a cuboid's rotation from its parallel projection
body: <p>I paint in my spare time, and that means I have a truly massive collection of reference images. Folders full of buildings, people, animals, cars, etc. It's gotten to the point where it'd be great to tag the objects by their pose, so I can find the right object at the right angle. CVAT, an image annotating tool for machine learning, allows you to mark images with cuboids, as you can see in this picture.</p>
  <p><a href="https://i.stack.imgur.com/6f1Sg.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/6f1Sg.png" alt="A cuboid annotation in CVAT" /></a></p>
  <p>But suddenly I'm wondering... is it even possible for a computer to estimate the rotation of a cuboid based on a single image, when all I can feed it are the eight (x,y) pairs that define the image of said cuboid?</p>
  <p>My thinking is that I need to somehow invert the transformation matrix so that this cuboid looks like a rectangle. That would mean that we're looking at it &quot;on-axis&quot;, and I'm imagining that this inversion could furnish me with those XYZ rotations I'm looking for.</p>
  <p>My best lead right now is OpenCv's <a href="https://theailearner.com/tag/cv2-getperspectivetransform/" rel="nofollow noreferrer">getPerspectiveTransform</a> function, which can create a matrix that will warp an image, but that transformation seems to be purely two-dimensional.</p>
  <p>Wikipedia does mention the idea of using an <a href="https://en.wikipedia.org/wiki/Affine_transformation#Augmented_matrix" rel="nofollow noreferrer">&quot;augmented matrix&quot;</a> to perform transformations in an extra dimension, which seems apropos here, since I want to go from a 2D representation to a 3d.</p>
  <p>A couple constraints &amp; advantages that might clarify the feasibility, here:</p>
  <ul>
  <li>The cuboids are rendered in a parallel projection. They don't match the perspective of the image, and that's okay! Just need a rough sense of their pose -- a margin of error of 10 degrees on any given axis of rotation is fine by me, in case there are some inexact solutions that could work.</li>
  <li>In the case of multiple cuboids in the scene, I don't care at all about their interrelations -- each case can be treated separately.</li>
  <li>I always have a sense of the &quot;rear wall&quot; of the cuboid, because I'm careful in how I make these annotations, in case that symmetry-breaking helps.</li>
  <li>The lengths of edges are irrelevant, I'm not trying to measure the &quot;aspect ratio&quot; of these  bounding cuboids.</li>
  </ul>
  <p>Thank you for any advice or hints!</p>