Union of Intersections (UoI) for Interpretable Data Driven Discovery and Prediction
The increasing size and complexity of scientific data could dramatically enhance discovery and prediction for basic scientific applications. Realizing this potential, however, requires novel statistical analysis methods that are both interpretable and predictive. We introduce Union of Intersections (UoI), a flexible, modular, and scalable framework for enhanced model selection and estimation. Methods based on UoI perform model selection and model estimation through intersection and union operations, respectively. We show that UoI-based methods achieve low-variance and nearly unbiased estimation of a small number of interpretable features, while maintaining high-quality prediction accuracy. We perform extensive numerical investigation to evaluate a UoI algorithm (U oI Lasso ) on synthetic and real data. In doing so, we demonstrate the extraction of interpretable functional networks from human electrophysiology recordings as well as accurate prediction of phenotypes from genotype-phenotype data with reduced features. We also show (with the U oI L1Logistic and U oI CU R variants of the basic framework) improved prediction parsimony for classification and matrix factorization on several benchmark biomedical data sets. These results suggest that methods based on the UoI framework could improve interpretation and prediction in data-driven discovery across scientific fields.
This paper focuses on model selection and, to some extent, feature selection in large datasets with many features, of which only a small subset are assumed to be necessary for accurate prediction.  The authors propose a general method by which model selection is performed by way of feature compression performed by taking the intersection of a multiple regularization parameters in an ensemble method, and then model estimation by taking a union over multiple outputs.  (This is also the major contribution of the paper.)  A second contribution is found in the union operation in a model averaging step with a boosting/bagging flavor.  Overall, I found the paper's method section well written and the idea proposed to be complete.  The paper's experimental section was difficult to follow, but the results do seem to support the framework.  One major missing part of the paper is a reasonable discussion of using the framework beyond a Lasso base.   Are there reasons why this method would not work for classification?  Are there potential hitches to using this method with already-ensemble-based methods like random forests?  While there are many uses for the UoI with a Lasso base already, it was increase the general interest of the framework if UoI could be placed in the more general ML space.