  Heterogeneous ensembles built from the predictions of a wide variety and
large number of diverse base predictors represent a potent approach to building
predictive models for problems where the ideal base/individual predictor may
not be obvious. Ensemble selection is an especially promising approach here,
not only for improving prediction performance, but also because of its ability
to select a collectively predictive subset, often a relatively small one, of
the base predictors. In this paper, we present a set of algorithms that
explicitly incorporate ensemble diversity, a known factor influencing
predictive performance of ensembles, into a reinforcement learning framework
for ensemble selection. We rigorously tested these approaches on several
challenging problems and associated data sets, yielding that several of them
produced more accurate ensembles than those that don't explicitly consider
diversity. More importantly, these diversity-incorporating ensembles were much
smaller in size, i.e., more parsimonious, than the latter types of ensembles.
This can eventually aid the interpretation or reverse engineering of predictive
models assimilated into the resultant ensemble(s).

Machine Learning;Machine Learning