Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction
Time series prediction problems are becoming increasingly high-dimensional in modern applications, such as climatology and demand forecasting. For example, in the latter problem, the number of items for which demand needs to be forecast might be as large as 50,000. In addition, the data is generally noisy and full of missing values. Thus, modern applications require methods that are highly scalable, and can deal with noisy data in terms of corruptions or missing values. However, classical time series methods usually fall short of handling these issues. In this paper, we present a temporal regularized matrix factorization (TRMF) framework which supports data-driven temporal learning and forecasting. We develop novel regularization schemes and use scalable matrix factorization methods that are eminently suited for high-dimensional time series data that has many missing values. Our proposed TRMF is highly general, and subsumes many existing approaches for time series analysis. We make interesting connections to graph regularization methods in the context of learning the dependencies in an autoregressive framework. Experimental results show the superiority of TRMF in terms of scalability and prediction quality. In particular, TRMF is two orders of magnitude faster than other methods on a problem of dimension 50,000, and generates better forecasts on real-world datasets such as Wal-mart E-commerce datasets.
This paper presents an approach to improve matrix factorization framework in the context of forecasting, where one of the matrix dimensions is time. The main contribution is the proposal of a regularization term that is the log-likelihood of a temporal model regarding some parameters to estimate. In fact, this is a funny way to put it. What their model is really doing is generating n time series by linearly combining k latent time series, governed by a Gauss-Markov model, and where the n weights of the linear combinations can be learned.  This approach has several benefits, first it can handle missing-value easily, second it can share information across different time-series while keeping temporal coherency in the model. Contrarily to graph-based regularization, the weights of edges do not have to be specified but can be learned and negative correlation are supported. The author instantiate this framework with a simple AR model. In this special case, they show how the temporal regularizer can be expressed as graph regularization which allow to use common solvers. Finally, they do comparison on 5 datasets with improvement in running time and around 10% improvement in accuracy compared to other methods. 
The paper is well written (except maybe I find the formulation of the time series part as "regularizer" overly complex, see below), and also well motivated. The proposed method does better than a range of competitors in experiments on a set of datasets, some of which are proprietary (which is fine with me). Finally, I find the application domain (retail forecasting) exciting and of high impact, and would like to see more work in the NIPS community in this direction (even though the lack of publicly available forecasting datasets of sufficient quality is clearly a hindrance).  On the other hand, I am somewhat disappointed by the overall scope of the work. In fact, what is proposed here is to model a smaller set of k dependent time series in a standard fashion (correlated AR), and then map these to n (partially) observed series using n linear weight vectors. That is all. Phrasing this as "regularization" is uncommon and does not help the understanding. I am not an expert on work in this area, but find it hard to accept that something like this has not been attempted before.  There is no mentioning of the fact that demand forecasting distributions are typically intermittent, and therefore poorly modelled by Gaussian noise (I recall an ICML paper by Chapados a few years ago that made this point). Also, there is no mentioning of basic structural elements, such as seasonality, promotions, or other features. For example, Walmart will sell more just before Christmas, and more turkey before Thanksgiving. In fact, what is described here, does not depend on other data than the demand targets at all. I doubt that a method like this would work very well on real demand data.  The second problem is the range of methods the authors compare against. Given that the proposed method is not very realistic as far as forecasting goes (no seasonality, no external inputs such as promotions, ...), it is understandable the authors compare against equally simple methods. But why not compare against simple things like Hyndman's forecast package, run separately on each series? This would take seasonality factors into account. The authors also do not try other basic setups in the ARIMA space. In fact, I am not convinced from these results that MF would in fact outperform an approach of individually forecasting each series, if only this was done with best practices. I have the strong feeling that a paper with this kind of evaluation would not pass at a forecasting conference or journal.  It also seems a bit unfair in my opinion to use only AR(1) in the comparison whereas domain knowledge was used to specify the lags of TRMF-AR. From the running time mentioned in Figure 5, at least AR({1,24}) for hourly grain and AR({1,52}) for weekly grain experiments can be used.   The scalability argument given by the author versus AR is also contrasted by the fact that AR is embarrassingly parallel and can be trained in multiple machine without difficulties allowing to scale to arbitrary large number whereas the proposed method does not enjoy this property.  Detailed comments:  p5: “unlike most approaches mentioned in section 2.2”. This sounds a bit vague to me, probably this refers only to Graph based approach. However in the second point, it says that the methods cant handle different lag which is not true for Graph based methods.   p6: please add Coefficient of Variation in the table to account for the forecast difficulty in Table 1.  p6: About Walmart dataset, the missing ratios indicated in Table 1 are not the initial one. The appendix mentions that time-series are aligned by adding unobserved data points. Please also add the proportion of unobserved data in the main document of the initial time-series, the proportion is probably much smaller and this gives the wrong impression if the reader does not check the appendix (if this ratio is very small then all methods could be tested in table 2 by replacing unobserved by zeros).  p7: although it can be understood in next page with the lag chosen for electricity and traffic datasets, it will help to specify the hourly time grain sooner, it is only stated explicitly in the appendix.  p7: Figure 4. the values of n (500, 1000, 5000, 10000, 50000) are not easy to read. W  p8: R-DLM should could be removed as it cannot handle any of the datasets except the articial one (maybe just mention that it only handles n lower than 32 in the text).