Comparing PCAs using the distances between the centroids of groups For example, I have a data for N countries where quantitative values for men and women are given. I ran PCA separately for the data from each country. PC1 and PC2 explain most of the variance, so I use them for further steps. I want to test how similar or different the groups - i.e. men and women - are, in respect to their separation or closeness on the PCA scatterplot (PC1 vs PC2). To do this, I calculated a quantitative proxy which is the (Euclidean) distance between the centroids of the two groups, as shown in the schematic (top). The two plots are some two countries.
I have a few questions about this approach to compare, in the context of the PCAs:

*

*Could the (Euclidean) distance between the centroids of the groups be used as an appropriate proxy to measure how similar/different the groups are?

*Can such a proxy be comparable across different PCAs? In the context of the example, because the distance for a country x is higher than country y, can I state that that men and women from country x are relatively different compared to country y?

*Finally, extending the pairwise comparisons, based on the distribution of the distances, can I make a statement that a country has higher or lower than average difference between men and women?



I have concerns with this methodology. Note that there's no guarantee that PC1 and PC2 have anything to do with gender differences. Seeing that men and women are no different in PC1 and PC2 does not indicate that men and women are identical, it just indicates there is no difference in the most variant dimensions. You may find very large differences in PC3, PC4, and beyond, despite the fact that those dimensions don't capture the most variance in the population as a whole.
The fact that you're computing a different PCA for every country makes distances in PCA space incomparable, since you are likely comparing men and women in different ways for every country. PC1 in country X may be a proxy for height in which case you would find differences between men and women, but PC1 in country Y may be a proxy for something like "number of toes", which probably doesn't vary much  by gender. Seeing a greater difference in PC1 for country X doesn't mean that men and women are more different in country X, it just means they are more different in whatever dimension happened to have the most variance.
You'd probably be best off computing a single principal component decomposition and applying it consistently to all countries. With that you, can evaluate the size of male vs. female differences along a particular dimension, and fairly compare those differences between countries.
So, to answer your specific questions: 1) Yes, distance in PCA space can be a proxy of distance in feature space, but note that your top PCAs may or may not have anything to do with gender differences. 2) No, comparing between different PCAs probably isn't meaningful since you are comparing differences on different dimensions. Seeing that men and women are different in the "apples" dimension in country X, and that they are not different in the "oranges" dimension in country Y allows you to make no statement about whether they are more or less alike overall in X or Y. 3) With consistent PCs across countries, you could evaluate which countries have the largest/smallest gender differences along a particular dimension.
