title: PerspectiveTransform not working properly
body: <p>I've an image which contains a chessboard image upon a rectangular object of interest. The image might be taken at any angle using a phone camera. Using the chessboard I'm trying to correct the perspective of the object of interest. But I'm not getting success with it.</p>
  
  <p>Here, is the output <a href="https://drive.google.com/open?id=1xSjm5Ww2Bn2TOvF8rWmHs8bIoGxmIiRt" rel="nofollow noreferrer">image</a>. If one draws a rectangle over the chessboard using photoshop, it can be seen that it's coming out to be a perfect rectangle but when a rectangle is fitted over the object, it's not perfect.</p>
  
  <p>Why is this happening? </p>
  
  <pre><code>rows = 9
  cols = 6
  criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS, 30, 0.001)
  flags = 0
  flags |= cv2.CALIB_CB_ADAPTIVE_THRESH
  flags |= cv2.CALIB_CB_FAST_CHECK
  flags |= cv2.CALIB_CB_NORMALIZE_IMAGE
  
  def calibrate(image):
      img = image.copy()
      objectPoints = np.zeros((rows * cols, 3), np.float32)
      objectPoints[:, :2] = np.mgrid[0:rows, 0:cols].T.reshape(-1, 2)
      objectPointsArray = []
      imgPointsArray = []
      h, w = img.shape[:2]
      gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
      # Find the chess board corners
      ret, corners = cv2.findChessboardCorners(gray, (rows, cols), flags=flags)
  
      # Make sure the chess board pattern was found in the image
      if ret:
          # Refine the corner position
          corners = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
  
          # Add the object points and the image points to the arrays
          objectPointsArray.append(objectPoints)
          imgPointsArray.append(corners)
  
          # Draw the corners on the image
          cv2.drawChessboardCorners(img, (rows, cols), corners, ret)
          plt.imshow(img)
          cv2.imwrite('out.jpg', img)
      ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objectPointsArray, imgPointsArray, gray.shape[::-1], None, None)
      for i in range(len(objectPointsArray)):
          imgPoints, _ = cv2.projectPoints(objectPointsArray[i], rvecs[i], tvecs[i], mtx, dist)
      return imgPoints, ret, mtx, dist, rvecs, tvecs
  
  imgPoints, ret, mtx, dist, rvecs, tvecs = calibrate(_roi)
  
  imgPoints = np.squeeze(imgPoints)
  p1 = imgPoints[0]
  p2 = imgPoints[8]
  p3 = imgPoints[-9]
  p4 = imgPoints[-1]
  
  pts = np.stack([p1,p2,p3,p4])
  screenCnt = order_points(pts)
  pts1 = np.float32(screenCnt)
  
  ratio=0.625
  cardH=math.sqrt((pts1[2][0]-pts1[1][0])*(pts1[2][0]-pts1[1][0])+(pts1[2][1]-pts1[1][1])*(pts1[2][1]-pts1[1][1]))
  cardW=ratio*cardH;
  pts2 = np.float32([[pts1[0][0],pts1[0][1]], [pts1[0][0]+cardW, pts1[0][1]], [pts1[0][0]+cardW, pts1[0][1]+cardH], [pts1[0][0], pts1[0][1]+cardH]])
  
  M = cv2.getPerspectiveTransform(pts1,pts2)
  
  transformed = np.zeros((img.shape[1], img.shape[0]), dtype=np.uint8);
  dst = cv2.warpPerspective(img, M, transformed.shape)
  </code></pre>