Answers.Body: <p>Assuming that the <span class="math-container">$6$</span> vertices of the hexagon are on the unit circle,</p>
  <pre><code>&gt;&gt;&gt; from sympy import *
  &gt;&gt;&gt; A = Matrix([[ 1, Rational(1,2),-Rational(1,2), -1, -Rational(1,2), Rational(1,2)], 
                  [ 0,     sqrt(3)/2,     sqrt(3)/2,  0,     -sqrt(3)/2,    -sqrt(3)/2]])
  &gt;&gt;&gt; A * A.T
  Matrix([[3, 0],
          [0, 3]])
  </code></pre>
  <p>Since <span class="math-container">${\bf A} {\bf A}^\top - 3 \, {\bf I}_2 = {\bf O}_2$</span>, <strong>any</strong> two orthogonal directions could be the principal components.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-30T00:30:44.460Z
Answers.Text: The 6 vertices of the hexagon are on the unit circle but the points on the second picture are not on the unit circle, yet for them $AA^T$ is also diagonal and is `Matrix([[5.25, 0],  [0, 5.25]])` with equal length diagonal elements. I wanted to use PCA to find direction of the largest variation of the data. Does it mean that PCA is not always find these directions even if data are not spherically symmetric? Also thanks for editing my question.
Answers.UserDisplayName: Vladislav Gladkikh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-30T09:05:20.117Z
Answers.Text: @VladislavGladkikh Rotate the point cloud by certain angles, and you obtain a point cloud that looks just like the one you started with, right? In my opinion, in these two cases, there's too much symmetry for a single "direction of the largest variation" to exist. At least in classical PCA. What if you use other norms?
Answers.UserDisplayName: Rodrigo de Azevedo
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-27T20:50:09.420Z
Answers.LastActivityDate: 2022-07-14T15:38:42.527Z
Answers.LastEditDate: 2022-07-14T15:38:42.527Z
Answers.UserDisplayName: Rodrigo de Azevedo
Body: <p>I do PCA on the data points placed in the corners of a hexagon, and get the following principal components:</p>
  <p><a href="https://i.sstatic.net/dXF3w.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/dXF3w.png" alt="enter image description here" /></a></p>
  <p>The PCA variance is <span class="math-container">$0.6$</span> and is the same for each component. Why is that? Shouldn't it be greater in the horizontal direction than in the vertical direction? The data is between <span class="math-container">$-1$</span> and <span class="math-container">$1$</span> in the <span class="math-container">$x$</span>-direction but only between <span class="math-container">$-\sqrt{3}/2$</span> and <span class="math-container">$\sqrt{3}/2$</span> in the <span class="math-container">$y$</span>-direction. Why PCA results in the equal length components?</p>
  <p>The length of each vector in the picture is the twice the square root of the variance.</p>
  <p>UPDATE: added more points, the variances changed to <span class="math-container">$0.477$</span> but still they are equal.</p>
  <p><a href="https://i.sstatic.net/Rg6Xs.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Rg6Xs.png" alt="enter image description here" /></a></p>
  <p>UPDATE 2: Added even more points, the variances changed to <span class="math-container">$0.44$</span> but still they are equal.</p>
  <p><a href="https://i.sstatic.net/DCxtU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DCxtU.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-31T00:28:13.623Z
Comments.Text: [Somewhat related](https://math.stackexchange.com/q/2375598/339790)
Comments.UserDisplayName: Rodrigo de Azevedo
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-31T02:43:07.690Z
Comments.Text: @RodrigodeAzevedo Yes, added more points, and updated my question. $AA^T$ remains diagonal. I wonder, what is the minimal symmetry at which PCA starts to fail to find the direction of largest variance.
Comments.UserDisplayName: Vladislav Gladkikh
Tags: <principal-component-analysis><dimensionality-reduction>
Title: Why does PCA of the vertices of a hexagon result in principal components of equal length?
UserDisplayName: Vladislav Gladkikh
ThreadText: # Why does PCA of the vertices of a hexagon result in principal components of equal length?
  
  (Asked by: username_0 on 2021-03-25)
  
  I do PCA on the data points placed in the corners of a hexagon, and get the following principal components:
  
  [![enter image description here](https://i.sstatic.net/dXF3w.png)](https://i.sstatic.net/dXF3w.png)
  
  The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be greater in the horizontal direction than in the vertical direction? The data is between $-1$ and $1$ in the $x$-direction but only between $-\sqrt{3}/2$ and $\sqrt{3}/2$ in the $y$-direction. Why PCA results in the equal length components?
  
  The length of each vector in the picture is the twice the square root of the variance.
  
  UPDATE: added more points, the variances changed to $0.477$ but still they are equal.
  
  [![enter image description here](https://i.sstatic.net/Rg6Xs.png)](https://i.sstatic.net/Rg6Xs.png)
  
  UPDATE 2: Added even more points, the variances changed to $0.44$ but still they are equal.
  
  [![enter image description here](https://i.sstatic.net/DCxtU.png)](https://i.sstatic.net/DCxtU.png)
  
  **Tags:** principal-component-analysis, dimensionality-reduction
  > **Comments:**
  > **username_1**: [Somewhat related](https://math.stackexchange.com/q/2375598/339790)  
  > **username_0**: @RodrigodeAzevedo Yes, added more points, and updated my question. $AA^T$ remains diagonal. I wonder, what is the minimal symmetry at which PCA starts to fail to find the direction of largest variance.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-27. Score: 3 | Accepted answer
  
  Assuming that the $6$ vertices of the hexagon are on the unit circle,
  >>> from sympy import *
  >>> A = Matrix([[ 1, Rational(1,2),-Rational(1,2), -1, -Rational(1,2), Rational(1,2)], 
                  [ 0,     sqrt(3)/2,     sqrt(3)/2,  0,     -sqrt(3)/2,    -sqrt(3)/2]])
  >>> A * A.T
  Matrix([[3, 0],
          [0, 3]])
  
  Since ${\bf A} {\bf A}^\top - 3 \, {\bf I}_2 = {\bf O}_2$, any two orthogonal directions could be the principal components.
  
  > **Comments:**
  > **username_0**: The 6 vertices of the hexagon are on the unit circle but the points on the second picture are not on the unit circle, yet for them $AA^T$ is also diagonal and is `Matrix([[5.25, 0],  [0, 5.25]])` with equal length diagonal elements. I wanted to use PCA to find direction of the largest variation of the data. Does it mean that PCA is not always find these directions even if data are not spherically symmetric? Also thanks for editing my question.  
  > **username_1**: @VladislavGladkikh Rotate the point cloud by certain angles, and you obtain a point cloud that looks just like the one you started with, right? In my opinion, in these two cases, there's too much symmetry for a single "direction of the largest variation" to exist. At least in classical PCA. What if you use other norms?  
  
  ---
  
  