PRUNE: Preserving Proximity and Global Ranking for Network Embedding
We investigate an unsupervised generative approach for network embedding. A multi-task Siamese neural network structure is formulated to connect embedding vectors and our objective to preserve the global node ranking and local proximity of nodes. We provide deeper analysis to connect the proposed proximity objective to link prediction and community detection in the network. We show our model can satisfy the following design properties: scalability, asymmetry, unity and simplicity. Experiment results not only verify the above design properties but also demonstrate the superior performance in learning-to-rank, classification, regression, and link prediction tasks.
The paper presents a NN model for learning graph embeddings that preserves the local graph structure and a global node ranking similar to PageRank. The model is based on a Siamese network, which takes as inputs two node embeddings and compute a new (output) representation for each node using the Siamese architecture. Learning is unsupervised in the sense that it makes use only of the graph structure. The loss criterion optimizes a weighted inner product of the output representations (local loss) plus a node ranking criterion (global loss). Some links with a community detection criterion are also discussed. The model is evaluated on a series of tasks: node ranking, classification and regression, link prediction, and compared to other families of unsupervised embedding learning methods.
The paper introduces new loss criteria and an original method for learning graph embeddings. The introduction of a node ranking term based on the graph structure is original. The authors find that this term brings interesting benefits to the learned embeddings and allows their method to outperform the other unsupervised baselines for a series of tasks. The links with the community criteria is also interesting. On the other hand, the organization and the form of the paper makes it somewhat difficult to follow at some places. The motivation for using a Siamese architecture is not explained. Two representations are learned for each node (notations u and z in the paper). Since the z representation is a deterministic function of the u representation, why are these two different representations needed, and why not directly optimizing the initial u representation? The explanation for the specific proximity preserving loss term is also unclear. For example, it is shown that this criterion optimizes a “second-order” proximity, but this term is never properly defined. Even if the result looks interesting, lemma 3.4 which expresses links with a community detection criterion is confusing.
In the experimental section, a more detailed description of the baselines is needed in order to understand what makes them different from the proposed method. The latter outperforms all the baselines on all the tests, but there is no analysis of the reasons for this good behavior.
Overall the paper reveals intriguing and interesting findings. The proposed method has several original aspects, but the paper requires a better organization/presentation in order to fully appreciate the proposed ideas.