The inference of models, prediction of future symbols, and entropy rate estimation of discrete-time, discrete-event processes is well-worn ground. However, many time series are better conceptualized as continuous-time, discrete-event processes. Here, we provide new methods for inferring models, predicting future symbols, and estimating the entropy rate of continuous-time, discrete-event processes. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with simple synthetic data, these new methods seem to be competitive with state-ofthe-art methods for prediction and entropy rate estimation as long as the correct model is inferred.
The paper focuses on the problem of modeling, predicting and estimating entropy information over continuous-time discrete event processes. Specifically, the paper leverages unifilar HSMM's for model inference and then uses the inferred states to make future predictions. The authors also use inferred model with previously developed techniques for estimating entropy rate. The authors describe the methods and provide the evidence of the effectiveness of their method with experiments on a synthetic dataset.
The paper should be rejected as it doesn't appear to be fit for this conference. While the paper does build techniques that leverages classical statistical learning and combine them with deep learning, it falls short on various aspects. Specifically, the paper neither provides any rigorous analysis of the developed methods so as to be useful to theoretical learning community nor does it support its approaches with strong empirical evidence so as to be of practical importance. 
- While the authors cite lack of work in such data setting (which is actually incorrect as described below), they do not motivate or justify why such an approach of using either unifilar HSMM or Bayesian inference techniques are good starting point to model such information. The authors claim that unifilar HSMM is more of a restriction to make the three proposed tasks easier. But such a model assumption is very strict and will likely lead to large model mis-specification issues when used in real-world. The author does not describe what can be done in this scenario.
- Page2, Section 3: the statistics of {(x_i, t_i)} are unchanging in time. What does this mean?
- As stated by authors, to make the approach tractable, they have to adopt a slew of approximations which seems to put severe restriction on what kind of data can be modeled and inferred on using these approaches.
- It is not clear why one wants to estimate the entropy rate in such settings and authors avoid discussing it in Section 4.2
similarly the authors explicitly avoid discussing bias-variance tradeoff of such method, however, this makes the paper
devoid of any insights into the proposed approach.
- How was k chosen to be 3 in Eq 10? 
- Another major miss for the paper is that there has been lot of work at the intersection of stochastic processes (e.g. temporal point processes) and machine learning techniques that aim to model exactly the continuous-time discrete event information and the paper does neither cites any of them nor discusses them or compare with them. We encourage the authors to look at  the relevant literature and position their work in comparison to these existing approaches. I provide a few references here as starting point, however, lot of follow-up work are available:
Point Processes and Machine Learning:
------------------------------------
Modeling the Dynamics of Learning Activity on the Web
Smart Broadcasting: Do You Want to be Seen?, Karimi et. al.
Uncovering the Temporal Dynamics of Diffusion Networks, Gomez-Rodriguez et. al.
Hidden semi-markov models:
-------------------------
Recurrent Hidden Semi-Markov Model, Dai et. al. 
Deep learning and temporal modeling:
-----------------------------------
Recurrent Marked Temporal Point Processes: Embedding Event History to Vector, Du et. al.
The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process, Mei et. al.
- Finally, the authors show only one synthetic experiment. This is far from providing convincing evidence of the efficacy of the method and certainly needs a lot more work for acceptance in any machine learning conference.  The authors, at the least, must use their approaches on couple of real-world dataset (e.g. earthquake information, animal behavior as they mention or others) and also compare with the above stochastic processes techniques to test how their methods fare. 
- Also, the analysis of efficiency of their method is required. It appears that their methods may not be scalable for instance to a multi-dimensional discrete label case.
- Figure 4 is not correctly referenced in the paragraph right above Section 4.3.