Graph Neural Networks (GNNs) broadly follow the scheme that the representation vector of each node is updated recursively using the message from neighbor nodes, where the message of a neighbor is usually pre-processed with a parameterized transform matrix. To make better use of edge features, we propose the Edge Information maximized Graph Neural Network (EIGNN) that maximizes the Mutual Information (MI) between edge features and message passing channels. The MI is reformulated as a differentiable objective via a variational approach. We theoretically show that the newly introduced objective enables the model to preserve edge information, and empirically corroborate the enhanced performance of MI-maximized models across a broad range of learning tasks including regression on molecular graphs and relation prediction in knowledge graphs.
This paper proposed an auxiliary loss based on mutual information for graph neural network. Such loss is to maximize the mutual information between edge representation and corresponding edge feature in GNN ‘message passing’ function. By assuming a gaussian distribution of edge feature given edge representation, the training can be done efficiently with tractable density. Experiments on molecule regression and knowledge graph completion show better performance than MPNN. 
Overall the paper is written in a clear way which is easy to follow. The idea of using mutual information as some kind of regularization is also interesting. However, there are some concerns I have with the paper:
Regarding formulation
1. The derivation up to Eq(8) looks fine to me, where the assumptions are reasonable. However from Eq(8) one can see this is reduced to an ‘auto-encoder’ type of regularization, where one can have a trivial solution for reconstruction -- the identity network, when the hidden dimension is larger than input dimension. And in this paper, dimension of W should always be larger than the dimension of e (for example, in molecules e should be low dimension vector with the bond type, distance, etc., while W should have dimension that matches the node embeddings).
I think the original loss (i.e., the supervised MSE, cross entropy etc) would help a bit with such degenerated case, but it is possible that the learned f(e) contains both identity mapping (or equivalent) and the representation that contributes to original loss.  
2. Actually I’m also not sure if I get the motivation here. If one needs to do this regularization for edges, why don’t we consider this auxiliary loss for node embeddings as well? As in molecules, atoms have more interesting features than bonds, which should account more if the mutual information loss is needed. 
Regarding experiment
1. In Figure 1, the training loss of EIGNN is better than MPNN. This is a bit counterintuitive to me, as I think the auxiliary is a kind of regularization -- which might help with generalization but not necessarily the training loss. 
2. The original paper of MPNN reports the relative MAE. Is it possible to report the results using the same metric as previous paper? It would make the comparison more consistent -- though showing the improvement in current way is not too bad. 
3. I think one simple ablation study would to concat the edge feature directly inside the ‘message passing’ procedure, or have some ‘residual’ type of connection for edge features.