title: Transform a frame to be as if it was taken from above using OpenCV
body: <p>I am working on a project for estimating a UAV (quadcopter) location using optical-flow technique. I currently have a code that is using <a href="http://docs.opencv.org/2.4/modules/video/doc/motion_analysis_and_object_tracking.html#calcopticalflowfarneback" rel="nofollow noreferrer">farneback</a> algorithm from OpenCV. The current code is working fine when the camera is always pointing to the ground.</p>
  
  <p>Now, I want to add support to the case when the camera is not pointing straight down - meaning that the quadcopter now has a pitch / roll / yaw (Euler angles). The quadcopters  Euler angles are known and I am searching for a method to compute and apply the transformation needed based on the known current Euler angles. So that the result image will be as if it was taken from above (see image below).</p>
  
  <p>I found methods that calculates the transformation when having 2 sets (source and destination) of 4 corners via <code>findHomography</code> or <code>getPerspectiveTransform</code> functions from OpenCV. But I couldn't find any methods that can do it with knowing only Euler angle (because I don't know the destination image corenrs).</p>
  
  <p>So my question is what method can I use and how in order to transform a frame to be as if it was taken from above using only Euler angles and camera height from ground if necessary?</p>
  
  <p>In order to demonstrate what I need:</p>
  
  <p><a href="https://i.stack.imgur.com/zwm4G.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zwm4G.jpg" alt="enter image description here"></a></p>
  
  <p>The relevant part of my current code is below:</p>
  
  <pre><code>for(;;)
  {
      Mat m, disp, warp;
      vector&lt;Point2f&gt; corners;
      // take out frame- still distorted
      cap &gt;&gt; origFrame;
      // undistort the frame using the calibration parameters
      cv::undistort(origFrame, undistortFrame, cameraMatrix, distCoeffs, noArray());
      // lower the process effort by transforming the picture to gray
      cvtColor(undistortFrame, gray, COLOR_BGR2GRAY);
  
      if( !prevgray.empty() )
      {
          // calculate flow
          calcOpticalFlowFarneback(prevgray, gray, uflow, 0.5, 3/*def 3 */, 10/* def 15*/, 3, 3, 1.2 /* def 1.2*/, 0);
          uflow.copyTo(flow);
  
          // get average
          calcAvgOpticalFlow(flow, 16, corners);
  
          // calculate range of view - 2*tan(fov/2)*distance
          rovX = 2*0.44523*distanceSonar*100;     // 2 * tan(48/2) * dist(cm)
          rovY = 2*0.32492*distanceSonar*100;     // 2 * tan(36/2) * dist(cm)
  
          // calculate final x, y location
          location[0] += (currLocation.x/WIDTH_RES)*rovX;
          location[1] += (currLocation.y/HEIGHT_RES)*rovY;
      }
      //break conditions
      if(waitKey(1)&gt;=0)
          break;
      if(end_run)
          break;
      std::swap(prevgray, gray);
  }  
  </code></pre>
  
  <h2>UPDATE:</h2>
  
  <p><strong>After successfully adding the rotation, I still need my image to be centered</strong> (and not to go outside of the frame window as shown below). I guess I need some kind of translation. <strong>I want the center of the source image to be at the center of the destination image.</strong> How can I add this as well?</p>
  
  <p>The rotation function that works:</p>
  
  <pre><code>void rotateFrame(const Mat &amp;input, Mat &amp;output, Mat &amp;A , double roll, double pitch, double yaw){
      Mat Rx = (Mat_&lt;double&gt;(3, 3) &lt;&lt;
                1,          0,           0,
                0, cos(roll), -sin(roll),
                0, sin(roll),  cos(roll));
      Mat Ry = (Mat_&lt;double&gt;(3, 3) &lt;&lt;
                cos(pitch), 0, sin(pitch),
                0, 1,          0,
                -sin(pitch), 0,  cos(pitch));
      Mat Rz = (Mat_&lt;double&gt;(3, 3) &lt;&lt;
                cos(yaw), -sin(yaw), 0,
                sin(yaw),  cos(yaw), 0,
                0,          0,           1);
  
      Mat R = Rx*Ry*Rz;
      Mat trans = A*R*A.inv();
  
      warpPerspective(input, output, trans, input.size());
  }
  </code></pre>
  
  <p>When I run it with <code>rotateFrame(origFrame, processedFrame, cameraMatrix, 0, 0, 0);</code> I get image as expected:  </p>
  
  <p><a href="https://i.stack.imgur.com/dgBei.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dgBei.jpg" alt="enter image description here"></a></p>
  
  <p>But when I run it with 10 degrees in roll <code>rotateFrame(origFrame, processedFrame, cameraMatrix, 20*(M_PI/180), 0, 0);</code>. The image is getting out of the frame window:  </p>
  
  <p><a href="https://i.stack.imgur.com/YTtI1.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/YTtI1.jpg" alt="enter image description here"></a></p>