Minimax Optimal Alternating Minimization for Kernel Nonparametric Tensor Learning
We investigate the statistical performance and computational efficiency of the alternating minimization procedure for nonparametric tensor learning. Tensor modeling has been widely used for capturing the higher order relations between multimodal data sources. In addition to a linear model, a nonlinear tensor model has been received much attention recently because of its high flexibility. We consider an alternating minimization procedure for a general nonlinear model where the true function consists of components in a reproducing kernel Hilbert space (RKHS). In this paper, we show that the alternating minimization method achieves linear convergence as an optimization algorithm and that the generalization error of the resultant estimator yields the minimax optimality. We apply our algorithm to some multitask learning problems and show that the method actually shows favorable performances.
This paper presents a new non-parametric tensor regression method based on kernels. More specifically, the authors proposed a regularization based optimization approach with alternating minimization for non-parametric tensor regression model [15]. Moreover, the theoretical guarantee for the proposed method is presented the paper. Through experiments on various datasets, the proposed method compares favorably with existing state-of-the-art.
This paper presents a new non-parametric tensor regression method based on kernels. More specifically, the authors proposed a regularization based optimization approach with alternating minimization for non-parametric tensor regression model [15]. Moreover, the theoretical guarantee for the proposed method is presented the paper. Through experiments on various datasets, the proposed method compares favorably with existing state-of-the-art.  The paper is clearly written and easy to read. I understand the key contribution of this paper is the theoretical analysis of the non-parametric tensor regression. However, the experimental section of this paper can be improved (See comments below). Overall, the proposed method is simple yet effective and theoretically guaranteed. I am happy to see the paper in NIPS.  Detailed comments: 1. In experiments, the performance of linear and nonlinear methods becomes similar when the number of samples increases. Are there any gap between linear and nonlinear methods with bigger sample size (e.g., 5000)? Also, including more comparison to other state-of-the-art linear tensor regression methods would be useful.  2. Related to the first question. If the performance of the linear and nonlinear methods are similar when the number of samples are large. Then, the experiments with web related data is not so practically interesting (though it is worth validating the method), since the web-data tends to be hundred millions of data points. That is, we might need to simply use a linear method for web-related datasets. Thus, if authors can include some experiments with high-dimensional and small number of samples datasets such as bioinformatics related datasets, the proposed method is more practically interesting. 3. How scale the AMP method with respect to the number of dimension? The method still works for d = 10^4 or more? 4. How the performance of the scaled latent convex regularization method? For completeness, it would be great to include it in either main text or supplementary materials. 5. How fast the AMP algorithm? There is no time comparison in Fig 1. (Author claimed the AMP method is faster in the main text without time comparison). 6. How to initialize the proposed method?