Statistical learning theory provides a formal criterion for learning a concept from examples. This theory addresses directly the trade-off in empirical fit and generalization. In practice, this leads to the structural risk-minimization principle where one minimizes a bound on the overall risk functional. For learning linear discriminant functions, this bound is impacted by the minimum of two terms---the dimension and the inverse of the margin. A popular and powerful learning mechanism, support vector machines, focuses on maximizing the margin. We compare this to methods that focus on minimizing the dimensionality, which, coincidentally, fulfills another useful criterion---the minimum description length principle.
['Haldun Aytug', 'Gary J. Koehler', 'Ling He']
Risk Minimization and Minimum Description for Linear Discriminant Functions