We propose Dynamically Pruned Message Passing Networks (DPMPN) for largescale knowledge graph reasoning. In contrast to existing models, embeddingbased or path-based, we learn an input-dependent subgraph to explicitly model reasoning process. Subgraphs are dynamically constructed and expanded by applying graphical attention mechanism conditioned on input queries. In this way, we not only construct graph-structured explanations but also enable message passing designed in Graph Neural Networks (GNNs) to scale with graph sizes. We take the inspiration from the consciousness prior proposed by Bengio (2017) and develop a two-GNN framework to encode input-agnostic full structure representation and learn input-dependent local one coordinated by an attention module. Experiments show the reasoning capability of our model to provide clear graphical explanations as well as predict results accurately, outperforming most stateof-the-art methods in knowledge base completion tasks.
The paper presents a graph neural network model inspired by the consciousness prior of Bengio (2017) and implements it by means of two GNN models: the inattentive and the attentive GNN, respectively IGNN and AGNN.
Introduction outlines clearly what is the goal and why this is in principle an interesting avenue to investigate, however I found it could be more precise. In particular message passing techniques can be implemented very efficiently and are highly scalable. Section on Cognitive intuition of the consciousness prior doesn’t tell much beyond the re-stating of what it is, I am more interested in what it brings to the table and on what is the intuition behind its application to this domain.
One thing I would argue agains is that the input-dependent local subgraph requires access to the global graph, therefore going back to a conventional GNN. The argument is that the message passing can be constrained to reduce the computational burden. However, this can also be done by means of anchor graphs or other data structures, dynamic pooling and so fort. How do the authors compare to such choices?
Overall I like the “global conditioning“ by means of sampling to have better local representations at each node, an idea that while it can be cast as consciousness prior it is also related to neural processes and architectures alike.
The implementation section could be made a bit clearer. Currently, for instance, there are references to prediction tasks while I think it would be nicer to have it fully self contained. Also, the aggregation strategy seems to be that of Kipf et al., is it a constraint of the model or other strategies could be used as well?
The experiment section is clearly explained and results are interesting, showing the potential of the proposed approach. Also the thorough analysis of the model is very well done.
I wonder what is the performance when using no sampling at all, shouldn’t this be the reference?
The convergence analysis states that the model converges very fast, does it also translate to better results in case of small amount of training data?