Wavelet regression and additive models for irregularly spaced data
We present a novel approach for nonparametric regression using wavelet basis functions. Our proposal, waveMesh, can be applied to non-equispaced data with sample size not necessarily a power of 2. We develop an efficient proximal gradient descent algorithm for computing the estimator and establish adaptive minimax convergence rates. The main appeal of our approach is that it naturally extends to additive and sparse additive models for a potentially large number of covariates. We prove minimax optimal convergence rates under a weak compatibility condition for sparse additive models. The compatibility condition holds when we have a small number of covariates. Additionally, we establish convergence rates for when the condition is not met. We complement our theoretical results with empirical studies comparing waveMesh to existing methods.
This paper proposes regression methods using wavelets, which does not require irregularly spaced data or the number of observation to be a power of 2. The key idea is to interpolate the raw data to fitted values on the regular grid and run regression algorithm with l1-penalty, i.e., proximal gradient descent. It is natural to generalize additive models for a potentially large number of covariates. The authors analyze (minimax) convergence rates of the proposed methods over Besov spaces. In experiments, they benchmark their methods under some synthetic functions and report that they perform better than competitors.  The necessary assumptions of traditional wavelets regressions (equi-spaced and a power of 2 data) restricts to apply them into practical applications. Therefore, it is nice work as authors overcome this issue. The analysis for convergence rates justifies to solve l1-penelty optimization. However, the interpolation idea is very natural to deal with irregular data points. More detail comparisons with other interpolating works would be required.  In addition, the experiments are only benchmarked with a power of 2 observations. This is not enough to demonstrate that the proposed methods are practically applicable. Authors should report the cases with general number of observations. In addition, although the experiments works well in some artificial functions (e.g., polynomial, sine and etc), a lack of benchmarks under real-world functions makes the paper weaken. It is worth to report additional experiments under real-world regression problems.   It would be better to understand if authors give more intuitions of interpolation schemes, e.g., construction of R in section 2.2, and how this can be computed by providing quantitative complexity in section 2.3. In addition, some readers may wonder how the linear interpolation matrix R is defined in Theorem 1.  The minor typos: In line 76: \phi_{jk}(x) = 2^{j/2} “”\phi””(2^j x - k) In line 143: O(K log k) -> O(K log K) In line 177: ca -> can