Factorization Machines (FMs) is an important supervised learning approach due to its unique ability to capture feature interactions when dealing with highdimensional sparse data. However, FMs assume each sample is independently observed and hence incapable of exploiting the interactions among samples. On the contrary, Graph Neural Networks (GNNs) has become increasingly popular due to its strength at capturing the dependencies among samples. But unfortunately, it cannot efficiently handle high-dimensional sparse data, which is quite common in modern machine learning tasks. In this work, to leverage their complementary advantages and yet overcome their issues, we proposed a novel approach, namely Deep Relational Factorization Machines, which can capture both the feature interaction and the sample interaction. In particular, we disclosed the relationship between the feature interaction and the graph, which opens a brand new avenue to deal with high-dimensional features. Finally, we demonstrate the effectiveness of the proposed approach with experiments on several real-world datasets.
In this paper, the authors propose generalize the FM to consider both interaction between features and interaction between samples. For the interaction between features, the authors propose to use graph convolution to capture high-order feature interactions. Moreover, the authors construct a graph on the instances based on similarity. Then a GCN is applied to the sample graph where the feature embedding is shared between the two components. Experiments are carried out on four datasets with tasks of link prediction and regression. Comparison to several baselines demonstrate the superior performance of the proposed method.
Strength:
1. The idea of utilizing GCN on the feature co-occurrence graph is interesting and innovative. The idea could possibly be combined with other variants of Deep FM models.
2. It is an interesting idea to combine sample similarity together with feature co-occurrence for better prediction accuracy.
Weakness:
1. Many descriptions in the paper are not very clear. First, the authors only mention how prediction is carried out with trained parameters. However, there is no description of the training process like what is the target used for the two components. What is the training procedure? Are the two components trained jointly? Second, the authors provide little description on how the sample similarity graph is constructed excepts for the Ad campaign dataset. Third, it is not clear how is the link prediction evaluation carried out. From the size of the graph, the authors seem to include both user and item in the graph. However, the user and item has disjointed feature set. It is not clear how the GCN is computed for the heterogenous nodes in the graph. Moreover, how is link prediction carried out, by taking inner product (cosine similarity) of the final representation.
2. For equation (8) in section 4.1, why we need to compute h_i^{RFI}. This should be the feature representation of sample i. However, the average is computed without include sample i itself. Also, are the neighbors defined in the sample similarity graph? Should we use the sample interaction in section 4.2 to capture that?
3. Though it is interesting idea to use graph convolution on the feature occurrence graph, it would be much better if the authors could provide more intuition on the output of the GCN. It would be helpful to study a few simple cases like without non-linearity. Is it a generalization to high-order FM without non-linearity? Also, it would be interesting to see experiments results using the graph convoluted feature representation directly for final representation. Also, some visualization of the learned feature embedding also helps.
4. The authors should carry out ablation study for different components of the model. Moreover, it would be much better if the authors can carry out experiments on some widely used recommendation datasets and use standard evaluation metrics for ranking.