Different kinds of representation learning techniques on graph have shown significant effect in downstream machine learning tasks. Recently, in order to inductively learn representations for graph structures that is unobservable during training, a general framework with sampling and aggregating (GraphSAGE) was proposed by Hamilton and Ying and had been proved more efficient than transductive methods on fileds like transfer learning or evolving dataset. However, GraphSAGE is uncapable of selective neighbor sampling and lack of memory of known nodes that've been trained. To address these problems, we present an unsupervised method that samples neighborhood information attended by co-occurring structures and optimizes a trainable global bias as a representation expectation for each node in the given graph. Experiments show that our approach outperforms the state-of-the-art inductive and unsupervised methods for representation learning on graphs.
This paper modifies the GraphSAGE on unsupervised inductive node embedding.
The authors propose to use the bi-attention architecture to sample
interesting nodes (instead of the uniform sampler in GraphSAGE), and to use a
global embedding bias matrix in the local aggregating functions. The method
showed improvements over GraphSAGE and other baselines on unsupervised
graph embeddings.
The proposition makes sense and the performance improvements are expected.
A major comment, however, is that that the proposed method is useful in very
restricted settings, and it is not clear how to generalize to
other applications which GraphSAGE can be applied on.
The overall technical contribution is incremental and
may not have enough novelty to be published in ICLR.
The technical representation is very poor, unorganized and not self-contained.
The paper cannot pass the threshold merely based on the way it is presented.
In the algorithms, please give the output besides the input. After the
algorithms, please remark on the computational and memory cost.
In algorithm 1, what is this function BIATT()? 
After algorithm 1, please describe this function as well as SAGE(). 
In the beginning of section 3, please describe the meaning of the
global bias matrix. In algorithm 1, if B is zero-initialized, why
does one need it as input?
Some of the equations are poor formatted (e.g. reduce_sum in page 5).
Please try to use rigorous mathematical formulations instead of "pseudo equations".
For example, re-write "One_hot(i)". In section 3.2, explain A_{gg}, etc.
use $\langle \rangle$ instead of $\alpha$.
There are many typos.