Stochastic Nonparametric Event-Tensor Decomposition
Tensor decompositions are fundamental tools for multiway data analysis. Existing approaches, however, ignore the valuable temporal information along with data, or simply discretize them into time steps so that important temporal patterns are easily missed. Moreover, most methods are limited to multilinear decomposition forms, and hence are unable to capture intricate, nonlinear relationships in data. To address these issues, we formulate event-tensors, to preserve the complete temporal information for multiway data, and propose a novel Bayesian nonparametric decomposition model. Our model can (1) fully exploit the time stamps to capture the critical, causal/triggering effects between the interaction events, (2) flexibly estimate the complex relationships between the entities in tensor modes, and (3) uncover hidden structures from their temporal interactions. For scalable inference, we develop a doubly stochastic variational Expectation-Maximization algorithm to conduct an online decomposition. Evaluations on both synthetic and real-world datasets show that our model not only improves upon the predictive performance of existing methods, but also discovers interesting clusters underlying the data.
To address these issues, we first formulate a new data abstraction, event-tensor, to preserve all the time stamps for multiway data. In an event-tensor, each entry comprises a sequence of interaction events rather than a numerical value. Second, we propose a powerful Bayesian nonparametric model to decompose event-tensors (Section 3). We hybridize latent Gaussian processes and Hawkes processes to capture various excitation effects among the observed interaction events, and the underlying complex relationships between the entities that participated in the events. Furthermore, we design a novel triggering function that enables discovering clusters of entities (or latent factors) in terms of excitation strengths. Besides, the triggering function allows us to flexibly specify the triggering range (say, via domain knowledge) to better capture local excitations and to control the trade-off to the computational cost. Finally, to handle data where both the tensor entries and interaction events are many, we derive a fully decomposed variational model evidence lower bound by using Poisson process superposition theorem and the variational sparse Gaussian process framework (Titsias, 2009). Based on the bound, we develop a doubly stochastic variational Expectation-Maximization algorithm to fulfill a scalable, online decomposition (Section 4).
For evaluation, we examined our model in both predictive performance and structure discovery. On three real-world datasets, our model often largely improves upon the prediction accuracy of the existing methods that use Poisson processes and/or time factors to incorporate temporal information. Simulation shows the latent factors estimated by our model clearly reflect the ground-truth clusters while by the competing methods do not. We further examined the structures discovered by our model on the real-world datasets and found many interesting patterns, such as groups of 911 accidents with strong associations, locations of townships that are apt to have consecutive accidents, and UFO shapes that are more likely to be sighted together (Section 6).
Post rebuttal: thank you for your answers! I would encourage authors to consider experiments with matrix completion with time stamps in the camera ready version and compare to baselines such as Factorization Machines or Neural Networks where time information is ignored. In my opinion this could help to draw the attention of researchers outside of the Bayesian community and increase the significance of the work.  This paper considers the problem of tensor factorization across time. Proposed model combines Hawkes process to accommodate temporal dependencies, with base rate dependent on a function of latent factors equipped with Gaussian Process prior. Gaussian Process has been previously considered in the tensor factorization domain and showed some advantages over more classical multilinear functions. However the addition of Hawkes process while maintaining advantages of GP appear novel to me and more elegant than previous discrete time approaches. Authors take Variational Inference pass, but deriving ELBO is not trivial for the proposed model and as a workaround authors decompose Hawkes process into multiple Poisson processes and introduce additional categorical ("cause") latent variable to arrive at manageable ELBO.  I think this is a good paper addressing a fairly sophisticated problem of stochastic tensor decomposition with a new model and some interesting inference decisions.  Combination of Hawkes process and GP was new to me and interesting to see. Experimental baselines appear strong and experiments demonstrate the advantage of modeling time component and using GP. Overall paper is written well.  On the downside, method is quite slow, but nonetheless experimental results are shown on several relatively big datasets. The other downside is the practical usefulness of the method. Clustering analysis does not appear particularly meaningful (looking at the Tables in the supplement, I can't really relate to the interpretation suggested in the paper), although perhaps the data is to blame (method seems to identify some structure in the toy simulation experiment). Have you tried matrix completion? For example, 100k MovieLens data (for predicting user-movie interactions instead of actual ratings)? I believe timestamp is often discarded, but maybe your model can perform well by taking it into account. It would be interesting to know if the method can compete with black box prediction approaches (i.e. neural networks or Factorization Machines). It is ok if not, since there are other merits, but it could make the story more complete and show limitations of the approach (overall, authors did not really discuss what are the limitations besides run-time).  Experiments show that GP + Hawkes process are better than GP or multilinear factorization with discrete time. What about using discrete time approach and GP instead of multilinear factorization. Is there any prior work that uses GP (for tensor decomposition) in temporal setting?  Typos in lines 55, 377.