Partition-wise Linear Models
Region-specific linear models are widely used in practical applications because of their non-linear but highly interpretable model representations. One of the key challenges in their use is non-convexity in simultaneous optimization of regions and region-specific models. This paper proposes novel convex region-specific linear models, which we refer to as partition-wise linear models. Our key ideas are 1) assigning linear models not to regions but to partitions (region-specifiers) and representing region-specific linear models by linear combinations of partitionspecific models, and 2) optimizing regions via partition selection from a large number of given partition candidates by means of convex structured regularizations. In addition to providing initialization-free globally-optimal solutions, our convex formulation makes it possible to derive a generalization bound and to use such advanced optimization techniques as proximal methods and decomposition of the proximal maps for sparsity-inducing regularizations. Experimental results demonstrate that our partition-wise linear models perform better than or are at least competitive with state-of-the-art region-specific or locally linear models.
The paper introduces a novel convex region-specific linear models called partition-wise linear model. It assigns linear models to partitions of the input space and linear combination of these partition-specific models define the region-specific linear models. This allows them to construct convex objective functions.  They optimize both the regions and predictors by using sparsity inducing structured penalties.  As a specific example of the partition-wise linear model, they propose a global and local residual model which uses one global linear weight vector and a set of partition-wise linear weights. They provide generalization bounds from which they conclude that they can use a large number of partition candidates with little risk of over fitting.  The algorithm for solving the optimization problem is based on the decomposition of proximal maps.  They efficiently compare their method to others emphasizing that their method is the only one that achieves: joint optimization of regions and region specific predictors, initialization-independent optimizers, and meaningful generalization bounds. Experiments demonstrate the their models perform better or as well as region-specific or locally linear models. Quality: Their claims are well supported both by theoretical analysis and experimental results. They carefully describe their new key ideas and derive generalization bounds.  Thorough experiments show that their methods perform well in comparison with other methods. Clarity: The paper is well-organized, but at times, it is not as clear as it could be. This is primarily due to awkward sentence structure and missing articles. Originality: The approach is a novel way to overcome the challenge of non-convex simultaneous optimization of region and region specific models. The related work is well referenced and effectively compared to their model.