DEEP GRAPH INFOMAX
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised manner. DGI relies on maximizing mutual information between patch representations and corresponding high-level summaries of graphs-both derived using established graph convolutional network architectures. The learnt patch representations summarize subgraphs centered around nodes of interest, and can thus be reused for downstream node-wise learning tasks. In contrast to most prior approaches to unsupervised learning with GCNs, DGI does not rely on random walk objectives, and is readily applicable to both transductive and inductive learning setups. We demonstrate competitive performance on a variety of node classification benchmarks, which at times even exceeds the performance of supervised learning.
encoder models based on graph convolutions (Gilmer et al., 2017), it is unclear whether randomwalk objectives actually provide any useful signal, as these encoders already enforce an inductive bias that neighboring nodes have similar representations.
In this work, we propose an alternative objective for unsupervised graph learning that is based upon mutual information, rather than random walks. Recently, scalable estimation of mutual information was made both possible and practical through Mutual Information Neural Estimation (MINE, Belghazi et al., 2018), which relies on training a statistics network as a classifier of samples coming from the joint distribution of two random variables and their product of marginals. Following on MINE,  introduced Deep InfoMax (DIM) for learning representations of highdimensional data. DIM trains an encoder model to maximize the mutual information between a high-level "global" representation and "local" parts of the input (such as patches of an image). This encourages the encoder to carry the type of information that is present in all locations (and thus are globally relevant), such as would be the case of a class label. DIM relies heavily on convolutional neural network structure in the context of image data, and to our knowledge, no work has applied mutual information maximization to graph-structured inputs. Here, we adapt ideas from DIM to the graph domain, which can be thought of as having a more general type of structure than the ones captured by convolutional neural networks. In the following sections, we introduce our method called Deep Graph Infomax (DGI). We demonstrate that the representation learned by DGI is consistently competitive on both transductive and inductive classification tasks, often outperforming both supervised and unsupervised strong baselines in our experiments.
This paper adapts the Deep Informax (DIM; Hjelm et al. 2018) method, which was used on
image data, into the graph domain. The architecture of the neural network and
the learning cost function are given by figure 1 and eq.(1), respectively.
The idea is to maximize the mutual information between a local representation
(of a "patch" defined by graph adjacency) and a global representation (of the entire graph),
so those different local patches are encouraged to carry some shared
global information.
This is in contrast to most unsupervised graph encoders, where the objective is
to fit the random walk similarities (node adjacency on the graph).
In an unsupervised learning scenario, where the graph structure and node features
are given, the authors achieved state-of-the-art performance on transductive and
inductive node classification tasks, in some cases even better than supervised baselines. 
The paper is well written. I recommend acceptance and have the following concerns.
Main comment 1
The title suggests that there are some information theory contents. 
However, section 3 does not include much information theory.
Rather, the author(s) directly give eq.(1) with pointers to references and informal discussions.
This is not so helpful. It is not straightforward for
the reader to relate eq.(1) with the definition of mutual information.
Ideally, before eq.(1) there should be one or two equations (with text)
to introduce the Jesen-Shannon MI estimation and information theoretic bounds etc.
Overall, due to this, the contribution is mainly on adapting the DIM method info the graph domain. Although the experimental results are good, there is not much theoretical insight or "recreative" introduction of the DIM method from the authors' perspectives. This is the main reason for that it is not a strong accept.
Main comment 2
A motivation of the proposition is to "not rely on random walks", or graph node adjacency.
Notice that random walks can be intuitively regarded as higher order node adjacency.
However, the encoder, which is based on GCN, does rely on the adjacency matrix,
as the convolution is done in local neighborhoods (that can also be defined based on
random-walk similarities). The authors are therefore suggested to make it
clear in related places that, it is the cost function which is not based
on node adjacency, although the neural network structure does rely on it.
As a related question, in the inductive experiments, in the mini-batch of 256 nodes
randomly selected, or selected by a local patch of the graph which is connected or nearby?
If it is the latter case, the cost function does rely on random-walk similarities,
as the summary vector will be a local patch average.
Questions:
-The summary vector is the average of all node features. On large graphs, the
average may carry less information as compared to small graphs. It can be
observed that on Pubmed and Reddit, the performance improvement is not as
high as the other small graphs. Could you comment on this?
-In the baseline "DeepWalk+features", are the two different types of features directly concatenated?
-Is it straightforward to apply DGI to link prediction tasks?
-It that a concern that the random corruption function will cause a high variance of the gradient?