Why is there a limit on how many principal components we can compute in PCA? In the PCA transformation below, you can see a dataset with 2 original features and a transformed version with two principal components. I understand why we can have fewer PCs, but why not more? What am I missing.


One way to think about this is via the orthogonality constraint imposed by PCA. From Wikipedia

The principal components of a collection of points in a real coordinate space are a sequence of $p$ unit vectors, where the $i$-th vector is the direction of a line that best fits the data while being orthogonal to the first $i − 1$ vectors.

Since you are working in two-dimensional space, there is no other possible direction for a new principal component that would be orthogonal to the first two.
