Kernel principal component analysis  In the field of multivariate statistics, kernel principal component analysis (kernel PCA)   is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are performed in a reproducing kernel Hilbert space. Recall that conventional PCA operates on zero-centered data; that is,  where formula_2 is the vector of one of the formula_3 multivariate observations. It operates by diagonalizing the covariance matrix, in other words, it gives an eigendecomposition of the covariance matrix: which can be rewritten as  To understand the utility of kernel PCA, particularly for clustering, observe that, while "N" points cannot, in general, be linearly separated in formula_7 dimensions, they can almost always be linearly separated in formula_8 dimensions. That is, given "N" points, formula_2, if we map them to an "N"-dimensional space with it is easy to construct a hyperplane that divides the points into arbitrary clusters. Of course, this formula_12 creates linearly independent vectors, so there is no covariance on which to perform eigendecomposition "explicitly" as we would in linear PCA. Instead, in kernel PCA, a non-trivial, arbitrary formula_12 function is 'chosen' that is never calculated explicitly, allowing the possibility to use very-high-dimensional formula_12's if we never have to actually evaluate the data in that space. Since we generally try to avoid working in the formula_12-space, which we will call the 'feature space', we can create the N-by-N kernel  which represents the inner product space (see Gramian matrix) of the otherwise intractable feature space. The dual form that arises in the creation of a kernel allows us to mathematically formulate a version of PCA in which we never actually solve the eigenvectors and eigenvalues of the covariance matrix in the formula_17-space (see Kernel trick). The N-elements in each column of "K" represent the dot product of one point of the transformed data with respect to all the transformed points (N points). Some well-known kernels are shown in the example below. Because we are never working directly in the feature space, the kernel-formulation of PCA is restricted in that it computes not the principal components themselves, but the projections of our data onto those components.