It is known that the solution of regularization and interpolation problems with Hilbertian penalties can be expressed as a linear combination of the data. This very useful property, called the rep- resenter theorem, has been widely studied and applied to machine learning problems. Anal- ogous optimality conditions have appeared in other contexts, notably in matrix regularization. In this paper we propose a unified view, which generalizes the concept of representer theorems and extends necessary and sufficient conditions for such theorems to hold. Our main result shows a close connection between representer theorems and certain classes of regularization penalties, which we call orthomonotone functions. This re- sult not only subsumes previous representer the- orems as special cases but also yields a new class of optimality conditions, which goes beyond the classical linear combination of the data. More- over, orthomonotonicity provides a useful crite- rion for testing whether a representer theorem holds for a specific regularization problem.
['Andreas A. Argyriou', 'Francesco Dinuzzo']
A Unifying View of Representer Theorems