Density-lines in PCA plot? Short version:
Someone wants me to draw "density-lines" onto a PCA plot for every point on the PCA plot. Which does not make sense in my understanding.
PCA is transforming the high dimensional data to principal components. How am I supposed to add a "density" for every point on a plot showing the two most relevant principal components?
Longer version:

*

*I have high-dimensional data. Gene expression data, to be precise (about n = 22000 dimensions). The data is from k biological samples, so every sample has values for the about 22'000 dimensions.

*Now I calculate the principal components and show the two principal components which explain most of the variation. The k samples now fall on the 2D plane reflecting the two most relevant principal components as they are calculated for the respective sample.

*My collaborator now wants "denisty-curves" on the PCA plot for every one of the k points. Which does not work for a PCA plot, right? We reduce the n-dimensional space to fewer dimensions and then project the data onto this (in my case) two principal components. So in my understanding density lines for the reduced data points do not make sense, do you agree?

Example -- Let's assume a PCA plot as in the first figure here: https://cran.r-project.org/web/packages/ggfortify/vignettes/plot_pca.html
Would you say that some sort of density lines for every point reflecting the underlying data does make sense in this case?

The data points are, after PCA transformation, distributed on the plane of the principal component axes.
That is an interpretation for which you can plot the density (ie. the density of the probability for a point to have a certain score on the principal components).
There are quite some examples with a search on the internet https://www.google.ch/search?q=pca+density+plot&source=lnms&tbm=isch
And in relation to your example with the iris dataset
https://www.google.ch/search?q=pca+density+iris&source=lnms&tbm=isch
I am linking to examples on Google because currently I have no R available to make a graph (maybe I will make an example myselve later). Below is an example from Fisher's article on the Iris data set. It is not PCA but LDA, and also it is not density lines but histograms/distributions. The principle in the sense that it is a projection onto a space of lower dimensions (in this case 1 dimension) and a display of the density distribution on that projected space.
"The use of multiple measurements in taxonomic problems" Annals of Eugenics, Vol VII, Pt. II, op. 179-188, 136


Marginal distribution
When you have multivariate data, where the multiple variables are distributed according to some joint distribution, then you can still think of the distribution of a single variable. This is also called a marginal distribution.
It is not different for PCA and a distribution for the principal component. You can view this as the shadow of a multidimensional joint distribution being projected onto the lower dimensional space. The distribution of the data in this lower dimensional space is like a marginal distribution.
See below an example of this projecting. The data is distributed in 3 dimensions (according to some density) but you can project the points into a lower space and following that think about the density in that lower space.


Images from this question: Interpreting PCA figures in layman terms
