In learning a predictive model over multivariate time series consisting of target and exogenous variables, the forecasting performance and interpretability of the model are both essential for deployment and uncovering knowledge behind the data. To this end, we propose the interpretable multi-variable LSTM recurrent neural network (IMV-LSTM) capable of providing accurate forecasting as well as both temporal and variable level importance interpretation. In particular, IMV-LSTM is equipped with hidden state matrix and update process, so as to learn variableswise hidden states. On top of it, we develop a mixture attention mechanism and associated summarization methods to quantify the temporal and variable importance in data. Extensive experiments using real datasets demonstrate the prediction performance and interpretability of IMV-LSTM in comparison to a variety of baselines. It also exhibits the prospect as an end-to-end framework for both forecasting and knowledge extraction over multi-variate data.
This paper describes a recurrent model (LSTM specifically, but generalizable) which can produce variable-wise hidden states that can be further used for two types of attentions: 1) variable importance for the importance of each variable (not accounting for time), and 2) temporal importance of each variable for the importance of each variable over time. The proposed NN model (IMV-LSTM) does not seem to directly provide such importance. Rather, the outputs are “decomposed” for each variable/time that allows probabilistic inference on top of this.
One of my main concerns (described in Cons/Comments below) is how it is not straightforward to grasp the quality of variable importance and temporal variable importance results despite this is the key strength of this paper. If this comes from my lack of understanding, I would appreciate if the authors could provide a little more explanation.
Pros:
1.	The overall quality of the paper is decent and mostly clear.
2.	The experiments are quite extensive.
3.	The fact that each variable should have different level of importance is interesting and practical.
Cons/Comments:
1.	The term “tensor” is used throughout the paper to describe the stacked matrices. While this is not technically wrong to describe 2>-dimensional structures, this term could potentially imply (and make the readers to expect) tensor-based schemes such as tensor decomposition. This is not necessarily bad, but to me, “tensor” and “variable-wise correspondence” do not seems to be associated too deeply since the “tensor” used in IMV-LSTM is a stack of matrices that are also independently used with respect to each other.
2.	The variable importance experiments seem quite extensive and thorough, especially the lists of variable-wise temporal importance matrices provided in the appendix. However, the authors could provide some significance or relevance of the findings with respect to any domain knowledge or literature, it may help further appreciate and interpret the quality of the variable importance which is quite subjective to non-experts. Such information may not even need to be in the main paper; including a short description in the appendix.
3.	Related to comments (2), the difference between IMV-Full and IMV-Tensor is hard to interpret since neither one is always better than the other (i.e., IMV-Full > IMV-Tensor in some experiments, vice versa). While the key difference is speculated to be from how the LSTM handles the variables, I am curious how this related to the differences in the results and how the differences variable importance results (i.e., Fig.3) can be in at least speculated.
Questions:
1.	Should \tilde{h}_t in Figure 1 (a) be \tilde{h}__{t-1} since this hidden state is from t-1? The figure itself currently implies that the hidden state for t is used, but this is computed from x_t using U_j. With \tilde{h}__{t-1}, it follows Eq.(1).
2.	In Equation set 2 for IMV-Tensor, are W and U (not W_j and U_j) also in tensor forms so each variable and hidden state get transformed correspondingly (i.e., W_1 for h^1_{t-1}, U_1 for x^1_t).
3.	The IMV-Tensor version of IMV-LSTM (related to the question above) can be considered as a set of parallel LSTMs, one for each variable. Such independence could also be inferred from Figure 1. If that’s the case, where do the variables “interact” with each other? Is this happening in the later stage where the hidden states across variable/time are aggregated in the attention stage (Eq.(8) and on)?
4.	Up until Eq.(8), n was used for the variable index where n = 1,…,N. In Eq.(8), it seems to be still used as the variable index (i.e., h_T^n and g^n), but it is also a set of possible values for a random variable z_{T+1}. Is n used the same way for z_{T+1} as well? I am slightly confused on how z is used. Also, (just to clarify), if we use N variables, we are using y_t as well (i.e., [x_t^1,…,x_t^{N-1}, y_t])?
5.	f_agg: Is this for aggregating over instances? For \bar{\alpha}^n, I’m guessing this is aggregated over instances for variable n for t=1,…,T_1.
6.	I am not too familiar with the notion of “time-lag” in the experiments. If the authors could explain this a little bit, I would appreciate it.