Reducing the Rank of Relational Factorization Models by Including Observable Patterns
Tensor factorization has become a popular method for learning from multirelational data. In this context, the rank of the factorization is an important parameter that determines runtime as well as generalization ability. To identify conditions under which factorization is an efficient approach for learning from relational data, we derive upper and lower bounds on the rank required to recover adjacency tensors. Based on our findings, we propose a novel additive tensor factorization model to learn from latent and observable patterns on multi-relational data and present a scalable algorithm for computing the factorization. We show experimentally both that the proposed additive model does improve the predictive performance over pure latent variable methods and that it also reduces the required rank -and therefore runtime and memory complexity -significantly.
The factorization rank which is also the number of latent variables is an important parameter in using matrix factorization methods. It affects both model predictive performance and runtime. Most importantly the scalability of most implemented methods for matrix factorization is limited by this parameter.This paper derives upper bound (number of strongly connected components) and lower bound (diclique partition number) on the required rank to recover adjacency tensors and determine the conditions under which factorization is an efficient approach for learning from relational data. The paper provides analysis on tensor rank and multi-linear rank. They provide three interesting insights from their analysis about efficiency of factorization methods with respect to the number of non-trivial connected components in relations, independence of rank and regularities between relations, and relation of dicliques with efficiency of factorization methods in comparison with common similarity metrics.Based on these findings an additive factorization model called additive relational effects model (ARE) is proposed. ARE augments RESCAL model with an additive term that holds the predictions of observable pattern methods.While the paper provides interesting insights and theoretical analysis seems valid, the use of oracle tensor in the evaluation is not very clear; When the ARE is defined the oracle tensor is formed by putting in predictions from heuristic similarity measures such as Jaccard or common neighbors. In the empirical evaluation however it is mentioned that a copy of observed tensor is used as M.  Yet, it is not clear if any of the similarity heuristics were used in construction of the oracle tensor in the evaluation section. If they were used, does the other compared methods perform factorization on the oracle tensor or the original observed one? In the former case, the performance boost of ARE could be due to benefiting mostly from the information of similarity heuristics, which is not a fair comparison. If they are not used, some performance analysis of including these measures in the oracle tensor could be discussed. The paper provides related arguments of this matter in section 3, and it would be beneficial to clearly discuss the effect of including them in the evaluation section.