title: 2D PCA line fitting with numpy
body: <p>I'm trying to implement a 2D PCA with numpy.
  The code is rather simple:</p>
  
  <pre><code>import numpy as np
  
  n=10
  d=10
  x=np.linspace(0,10,n)
  y=x*d
  
  covmat = np.cov([x,y])
  print(covmat)
  
  eig_values, eig_vecs = np.linalg.eig(covmat)
  largest_index = np.argmax(eig_values)
  largest_eig_vec = eig_vecs[largest_index]
  </code></pre>
  
  <p>The covariance matrix is:</p>
  
  <pre><code>[[   11.31687243   113.16872428]
   [  113.16872428  1131.6872428 ]]
  </code></pre>
  
  <p>Then I've got a simple helper method to plot a line (as a series of points) around a given center, in a given direction.
  This is meant to be used by pyplot, therefore I'm preparing separate lists for the x and y coordinate.</p>
  
  <pre><code>def plot_line(center, dir, num_steps, step_size):
      line_x = []
      line_y = []
      for i in range(num_steps):
          dist_from_center = step_size * (i - num_steps / 2)
          point_on_line = center + dist_from_center * dir
          line_x.append(point_on_line[0])
          line_y.append(point_on_line[1])
      return (line_x, line_y)
  </code></pre>
  
  <p>And finally the plot setup:</p>
  
  <pre><code>lines = []
  mean_point=np.array([np.mean(x),np.mean(y)])
  lines.append(plot_line(mean_point, largest_eig_vec, 200, 0.5))
  
  import matplotlib.pyplot as plt
  
  fig = plt.figure()
  ax = fig.add_subplot(111)
  
  ax.scatter(x,y, c="b", marker=".", s=10
             )
  for line in lines:
      ax.plot(line[0], line[1], c="r")
  
  ax.scatter(mean_point[0], mean_point[1], c="y", marker="o", s=20)
  
  plt.axes().set_aspect('equal', 'datalim')
  plt.show()
  </code></pre>
  
  <p>Unfortunately, the PCA doesn't seem to work.
  Here's the plot:</p>
  
  <p><a href="https://i.stack.imgur.com/SJyp4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SJyp4.png" alt="pca line fitting"></a></p>
  
  <p>I'm afraid I've got no idea what went wrong.</p>
  
  <ul>
  <li>I've computed the covariance manually -> same result.</li>
  <li>I've checked the other eigenvalue -> perpendicular to the red line.</li>
  <li>I've tested plot_line with the direction (1,10). It's perfectly aligned to my points:
  <a href="https://i.stack.imgur.com/iOBGe.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iOBGe.png" alt="perfectly aligned"></a></li>
  </ul>
  
  <p><strong>The final plot shows that the line fitted by pca is the correct result only it is mirrored at the y axis.</strong></p>
  
  <p>In fact, if I change the x coordinate of the eigenvector, the line is fitted perfectly:</p>
  
  <p><a href="https://i.stack.imgur.com/uKWI3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/uKWI3.png" alt="perfect fit"></a></p>
  
  <p>Apparently this is a fundamental problem. Somehow I've misunderstood how to use pca.</p>
  
  <p>Where is my mistake ?
  Online resources seem to describe PCA exactly as I implemented it.
  I don't believe I have to categorically mirror my line-fits at the y-axis. It's got to be something else.</p>