Graph representation learning for hypergraphs can be used to extract patterns among higher-order interactions that are critically important in many real world problems. Current approaches designed for hypergraphs, however, are unable to handle different types of hypergraphs and are typically not generic for various learning tasks. Indeed, models that can predict variable-sized heterogeneous hyperedges have not been available. Here we develop a new self-attention based graph neural network called Hyper-SAGNN applicable to homogeneous and heterogeneous hypergraphs with variable hyperedge sizes. We perform extensive evaluations on multiple datasets, including four benchmark network datasets and two single-cell Hi-C datasets in genomics. We demonstrate that Hyper-SAGNN significantly outperforms the state-of-the-art methods on traditional tasks while also achieving great performance on a new task called outsider identification.
Hyper-SAGNN will be useful for graph representation learning to uncover complex higher-order interactions in different applications.
Summary:
This work is tackling the problem of predicting variable-sized heterogeneous hyperedges in hypergraphs. The authors do so by introducing a multi-head attention mechanism that takes in a set of node features and predicts the probability of this set of nodes forming a hyperedge.
- Quality and clarity: 
The quality of the paper is good, the writing is clear and the reasoning is relatively easy to follow. The problem the authors are trying to tackle is also clearly defined. The motivation of the dynamic and static embedding choice could be explained better. The intuition that the pseudo-distance tells us how well the static embedding
of a node can be approximated by the features of the neighbour within the tuple somewhat makes sense, but I also have the impression that this is just one possible interpretation and what happens in practice could actually be quite different. A more formal treatment and analysis of this intuition would be very helpful.
- Originality: 
The main contribution here is to propose a more flexible formulation to improve over the fixed MLP used by DHNE. This in itself is a good idea, but overall is an incremental improvement.
- Significance: 
The results are convincing, DHNE is clearly outperformed on the benchmarks. Depending on the ease of training the model compared to DHNE, this work might be a useful contribution for the ICLR community.
Other points
The related work section should be expanded by discussing relationships to:
Bai, Song, Feihu Zhang, and Philip HS Torr. "Hypergraph Convolution and Hypergraph Attention." arXiv preprint arXiv:1901.08150 (2019).
Feng, Yifan, et al. "Hypergraph neural networks." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. 2019.
------------------
Post-rebuttal
------------------
I would like to thank the authors for their in-depth response. I found the additional analysis on the dynamic node embeddings insightful and it is reassuring that it confirms the claims. Overall I think the rebuttal addresses my major concerns and I will raise my score.