Random Projection Filter Bank for Time Series Data
We propose Random Projection Filter Bank (RPFB) as a generic and simple approach to extract features from time series data. RPFB is a set of randomly generated stable autoregressive filters that are convolved with the input time series to generate the features. These features can be used by any conventional machine learning algorithm for solving tasks such as time series prediction, classification with time series data, etc. Different filters in RPFB extract different aspects of the time series, and together they provide a reasonably good summary of the time series. RPFB is easy to implement, fast to compute, and parallelizable. We provide an error upper bound indicating that RPFB provides a reasonable approximation to a class of dynamical systems. The empirical results in a series of synthetic and real-world problems show that RPFB is an effective method to extract features from time series.
This paper proposes a new algorithm for learning on time series data. The basic idea is to introduce random projection filter bank for extracting features by convolution with the time series data. Learning on time series data can then be performed by applying conventional machine learning algorithms to these features. Theoretical analysis is performed by studying both the approximation power and statistical guarantees of the introduced model. Experimental results are also presented to show its effectiveness in extracting features from time series.
Major comments:
  (1) It seems that the algorithm requires the information on both $M$ and $\epsilon_0$, which depend on the unknown target function $h^*$ to be estimated. Does this requirement on an apriori information restrict its practical application? Since $Z_i'$ are complex numbers, the hypothesis function in (9) would take complex outputs. It is not clear to me how this complex output is used for prediction.
  (2) It is claimed that the discrepancy would be zero if the stochastic process is stationary. However, this seems not to be true as far as I can see. Indeed, the discrepancy in Definition 1 is a random variable even if the stochastic process is i.i.d.. So, it depends on the realization of $X$. If this is true, Theorem 1 is not clear to understand since it involves a discrepancy which would not be zero in the i.i.d. learning case.
  (3) There exists some inconsistences in Algorithm 1 and materials in Section 4. Algorithm 1 is a regularization scheme while ERM is considered in Section 4. There are $m$ time series in Algorithm 1, while $m=1$ is considered in Section 4. The notation is not consistent, i.e., $f$ is used in Algorithm 1 and $h$ is used in Section 4.
  (4) The motivation in defining features in Section 3 is not clear to me. It would be better if the authors can explain it more clearly so that readers without a background in complex analysis can understand it well.
Minor comments:
  (1) line 28: RKSH
  (2) line 65, 66: the range of $f\in\mathcal{F}$ should be the same as the first domain of $l$
  (3) eq (11), the function space in the brace seems not correct
  (4) the notation * in Algorithm 1 is not defined as far as I can see
  (5) $\delta_2$ should be $\delta$ in Theorem 1
  (6) line 254: perhaps a better notation for feature size is $n$?
  (7) Appendix, second line in equation below line 444, a factor of $2$ is missed there?
  (8) Appendix, eq (28), the identity should be an inequality? I suggest the authors to pay more attention to (28) since the expectation minus its empirical average is not the same as the empirical average minus expectation.