  In high-dimensional prediction problems, where the number of features may
greatly exceed the number of training instances, fully Bayesian approach with a
sparsifying prior is known to produce good results but is computationally
challenging. To alleviate this computational burden, we propose to use a
preprocessing step where we first apply a dimension reduction to the original
data to reduce the number of features to something that is computationally
conveniently handled by Bayesian methods. To do this, we propose a new
dimension reduction technique, called iterative supervised principal components
(ISPC), which combines variable screening and dimension reduction and can be
considered as an extension to the existing technique of supervised principal
components (SPCs). Our empirical evaluations confirm that, although not
foolproof, the proposed approach provides very good results on several
microarray benchmark datasets with very affordable computation time, and can
also be very useful for visualizing high-dimensional data.

Methodology