Consider a world in which events occur that involve various entities. Learning how to predict future events from patterns of past events becomes more difficult as we consider more types of events. Many of the patterns detected in the dataset by an ordinary LSTM will be spurious since the number of potential pairwise correlations, for example, grows quadratically with the number of events. We propose a type of factorial LSTM architecture where different blocks of LSTM cells are responsible for capturing different aspects of the world state. We use Datalog rules to specify how to derive the LSTM structure from a database of facts about the entities in the world. This is analogous to how a probabilistic relational model (Getoor & Taskar, 2007) specifies a recipe for deriving a graphical model structure from a database. In both cases, the goal is to obtain useful inductive biases by encoding informed independence assumptions into the model. We specifically consider the neural Hawkes process, which uses an LSTM to modulate the rate of instantaneous events in continuous time. In both synthetic and real-world domains, we show that we obtain better generalization by using appropriate factorial designs specified by simple Datalog programs.
Summary: 
 
The paper proposed to use Datalog rules to specify the design of the LSTM architecture for event data in continuous time. The LSTM module will be used to model the rate of the events. By incorporating Datalog rules, the paper aims to encode informed inductive biases into the model. 
Comments:
    After reading the entire paper, I think the main idea of this paper is to use sparse and structured weight matrices (called structural zeros” in the paper) to substitute the dense weight matrices in the original LSTM, and to split the hidden state into blocks where each block refer to a different world’s state.  
         How to design the structured weight matrices and how to define the node blocks, it is informed by the  Datalog rules. This design, however, will lead to a huge weight matrix and a very  long hidden state once the types of events and number of entities grow. The proposed model will face a severe scalability issue. From this point of view, only “structural zeros” weight matrices are not enough for an elegant model.                
 
        How to smartly share parameters and how to control the number of parameters will be an interesting direction to explore. This submission touches on this a bit but not in a principled way. For example, in Eq. 18(a) and 18(b), the embedding vectors for the grounded predicate is a summation of the embedding vectors of the entities and the predicates. This final embedding is empirically validated or is based on some permutation invariant property? This needs more clarification or some references.
 
 
        2.      The presentation needs to be polished. The current writing is not easy to follow. Especially for section 3. The architecture design needs to be clarified more. When I read this part, I felt a little difficult to map the Datalog rules to your model. 
        3        Since you are learning the vector embeddings for event types and entities, what are the advantages of this compared to the marked point process model, where the event types and entities are treated as discrete markers and are a much more parsimonious model. The Datalog rules can also be defined on the marker level by introducing a structured dependency structure over the markers. What are the potential benefits of learning the embeddings? The explanation is missing in this paper. 
       4     Lack of references. The proposed neural-symbolic architecture shares some similarities to the following papers: 
          (1) End-to-End Differentiable Proving 
          (2) DeepProbLog: Neural Probabilistic Logic Programming 
          (3) Neural Logic Machines.
          
        What are your contributions and differences in terms of the neural-symbolic architecture design?
 
       As for introducing logic rules to guide event predication, this is not a new topic. Here is a list of references:
 
         (1) PEL-CNF: Probabilistic event logic conjunctive normal form for video interpretation. 
          (2) A general framework for recognizing complex events in Markov logic. 
          (3) Learning Bayesian networks for clinical time series analysis. 
          (4) Logical Hierarchical Hidden Markov Models for Modeling User Activities. 
          (5) Slice Normalized Dynamic Markov Logic Networks. 
       5.         Lack of strong baselines. The paper only did a small-scale experiment study. It only compares a neural Hawkes process model. The experimental evaluation also needs stronger baselines. Specifically, methods that can handle continuous-time (e.g. marked point process) or probabilistic logic methods that can discretize time (as mentioned in the above references). The baselines are not quite strong and appear a bit arbitrary in the paper.