Online Learning for Multivariate Hawkes Processes
We develop a nonparametric and online learning algorithm that estimates the triggering functions of a multivariate Hawkes process (MHP). The approach we take approximates the triggering function f i,j (t) by functions in a reproducing kernel Hilbert space (RKHS), and maximizes a time-discretized version of the log-likelihood, with Tikhonov regularization. Theoretically, our algorithm achieves an O(log T ) regret bound. Numerical results show that our algorithm offers a competing performance to that of the nonparametric batch learning algorithm, with a run time comparable to parametric online learning algorithms.
This paper describes an algorithm for optizimization of Hawkes process parameters in on-line settings, where non-parametric form of a kernel is learnt. The paper reports a gradient approach to optimization, with theoretical analysis thereof. In particular, the authors provide: a regret bound, justification for simplification steps (discretization of time and truncation of time over which previous posts influence a new post), an approach to a tractable projection of the solution (a step in the algorithm), time complexity analysis.
The paper is very well written, which is very helpful given it is mathematically involved. I found it tackling an important problem (on-line learning is important for large scale datasets, and non-parametricity is a very reasonable setting when it is hard to specify a reasonable kernel form a priori). The authors provide theoretical grounding for their approach. I enjoyed this paper very much and I think it is going to be influencial for practical application of Hawkes processes.
Some comments:
- Can you comment on the influence of hyperparameter values (regularization coefficients, potentially also discretization step)? In experiments you set them a-priori to some fixed values, and it is not uncommon in the experiments that values of regularization hyperparameters have a huge influence on the experiments.
- Your experiments are quite limited, especially the synthetic experiments where many values are fixed and a single matrix of influence forms is considered. I think it would be much more compelling if some simulation was considered, where values would be repeatedly drawn. Also, why do you think this specific setting is interesting/realistic?
- I suggest making the font in Figures larger (especially for Figure 2 it is too small), it doesn't look good now. This is especially annoying in the supplementary (Figure 4).