iclr2019_conference
There are time series that are amenable to recurrent neural network (RNN) solutions when treated as sequences, but some series, e.g. asynchronous time series, provide a richer variation of feature types than current RNN cells take into account. In order to address such situations, we introduce a unified RNN that handles five different feature types, each in a different manner. Our RNN framework separates sequential features into two groups dependent on their frequency, which we call sparse and dense features, and which affect cell updates differently. Further, we also incorporate time features at the sequential level that relate to the time between specified events in the sequence and are used to modify the cell's memory state. We also include two types of static (whole sequence level) features, one related to time and one not, which are combined with the encoder output. The experiments show that the proposed modeling framework does increase performance compared to standard cells.
[Relevance] Is this paper relevant to the ICLR audience? yes
[Significance] Are the results significant? somewhat
[Novelty] Are the problems or approaches novel? reasonably
[Soundness] Is the paper technically sound? yes, I think. I did not thoroughly check the equations.
[Evaluation] Are claims well-supported by theoretical analysis or experimental results? somewhat
[Clarity] Is the paper well-organized and clearly written? yes, except the experiments
Confidence: 2/5
Seen submission posted elsewhere: No (but I did find it on arXiv after writing the review)
Detailed comments:
In this work, the authors propose a new type of memory cell for RNNs which account for multiple types of time series. In particular, the work combines uniformly sampled observations (“normal” RNN input), time-decaying observations, non-decaying observations which may change, and static features which are not time-dependent. Empirically, the proposed approach outperforms an RNN with TLSTM cells in some cases.
=== Comments
I found the proposed approach for incorporating the different types of time series reasonable. This work definitely leans heavily on ideas from TLSTM and others, but, to the best of my knowledge, the specific combination and formulation is novel, especially concerning the “non-decaying time series” observations.
 However, I had difficult coming up with an example of a “static decay feature”. It would be helpful to give a concrete example of one of these in the main text. (It is also not clear to me why the difference in time between the time of the last event in a sequence and the prediction time for that sequence would be considered a “static decay” feature rather than just a “static” feature.)
My main concern with the paper is that the experimental design and results are not especially easy to follow; consequently, they are not as convincing as they might be. First, the sparsity mechanism is rather simple. In many domains (e.g., the medical domain considered in several of the cited papers), missingness is non-uniform and is often meaningful. While “meaningfulness” may be difficult to simulate, burstiness (non-uniformity) could be simulated. Second, for the groups, it is not clear whether all combinations of, e.g., 2 (informative) feature were sparsely sampled or if only one group of 2 was chosen. If the former, then some measure of variance should be given to help estimate statistical significance. Third, the particular classification task here is, essentially, forecasting one of the input variables. While that is certainly a relevant problem, many other time series classification or regression problems are not tied so directly to observations. It is not clear if these results are relevant in that setting.
=== Typos, etc.
The plots and figures in the paper are very difficult to read. Larger versions, or at least versions with increased fonts, should be used.