Granger causality is a widely-used criterion for analyzing interactions in largescale networks. As most physical interactions are inherently nonlinear, we consider the problem of inferring the existence of pairwise Granger causality between nonlinearly interacting stochastic processes from their time series measurements. Our proposed approach relies on modeling the embedded nonlinearities in the measurements using a component-wise time series prediction model based on Statistical Recurrent Units (SRUs). We make a case that the network topology of Granger causal relations is directly inferrable from a structured sparse estimate of the internal parameters of the SRU networks trained to predict the processes' time series measurements. We propose a variant of SRU, called economy-SRU, which, by design has considerably fewer trainable parameters, and therefore less prone to overfitting. The economy-SRU computes a low-dimensional sketch of its high-dimensional hidden state in the form of random projections to generate the feedback for its recurrent processing. Additionally, the internal weight parameters of the economy-SRU are strategically regularized in a group-wise manner to facilitate the proposed network in extracting meaningful predictive features that are highly time-localized to mimic real-world causal events. Extensive experiments are carried out to demonstrate that the proposed economy-SRU based time series prediction model outperforms the MLP, LSTM and attention-gated CNN-based time series models considered previously for inferring Granger causality.
In this paper the authors propose using Statistical Recurrent Units to predict the network for Granger causality. They motivate this choice by the high representation power of SRUs for multivariate time series, the good performance they usually enjoy and as a way to alleviate the vanishing gradient problem. More importantly the particular form of the SRUs gives a very simple predictor and therefore explanability for Granger causality: the authors propose to simply mark serie $i$ as Granger caused by $j$ if the $j$th column of the input mixing matrix of the $g_i$ is non-zero.
In order to force whole columns of the input matrix to be negative, the authors use a group regularization on the columns of the input matrix $W_{\text{in}}$. The resulting problem is then optimized by proximal gradient descent.
The second contribution of the authors is eSRU, a smaller (in terms of parameters) variant of SRU in order to prevent overfitting.
First the authors introduce a dimensionality reduction layer before the SRU units by using fixed non-trainable random projections. Then the authors propose adding a regularization term for the output mixing matrix, which represents the bulks of the parameters of the model.
The eSRU and SRU causal models presented are then compared to the previous state of the art on several datasets, both synthetic and real, where they manage to reach a new state of the art.
Regarding the two improvements in eSRU proposed by the authors, I have several questions:
First, regarding the random projections, have the authors tried learning the projections ? While it increases the risk of overfitting it is possible, with enough regularization, that it may help learn good representations. Secondly, as I understand it a single projection is drawn at random for every component. While we know from Johnson-Lindenstrauss's Lemma that this will in average be a good strategy, how stable is the model to spurious projections ?
Regarding the scarification of the output matrix: the number of parameters is effectively reduced but the computational requirements are still the same. Did the authors try any explicit methods, maybe matrix factorization, to exploit the very specific sparse structure of $W_o$ and reduce the number of operations and parameters ?
Finally, while the results are undeniable, just observing the non-zero columns of $W_\text{in}$ is as I understand it a mostly ad hoc rule. It would have been interesting to empirically verify its validity by contrasting it with the results given by following equation 2.
Overall the paper presents an interesting model for inferring Granger causality. The authors clearly present their two main contributions and verify them using varied datasets.
The authors also made a clear and appreciated effort toward reproducibility by including all hyperparameters and implementation details as well as the code, including those of competing techniques.
The inclusion of the literature review in the appendix is also greatly appreciated.