DFacTo: Distributed Factorization of Tensors
We present a technique for significantly speeding up Alternating Least Squares (ALS) and Gradient Descent (GD), two widely used algorithms for tensor factorization. By exploiting properties of the Khatri-Rao product, we show how to efficiently address a computationally challenging sub-step of both algorithms. Our algorithm, DFacTo, only requires two sparse matrix-vector products and is easy to parallelize. DFacTo is not only scalable but also on average 4 to 10 times faster than competing algorithms on a variety of datasets. For instance, DFacTo only takes 480 seconds on 4 machines to perform one iteration of the ALS algorithm and 1,143 seconds to perform one iteration of the GD algorithm on a 6.5 million × 2.5 million × 1.5 million dimensional tensor with 1.2 billion non-zero entries.
By exploiting the properties of the Khatri-Rao product, this paper proposes a technique to speed up Alternating Least Squares (ALS) and Gradient Descent (GD) algorithms for tensor factorization. Furthermore, parallel algorithms are also proposed for scalability. Experiments on large-scale tensors are used for evaluation.The problem of distributed (parallel) tensor factorization is interesting and challenging. The proposed (centralized) technique for speeding up ASL and GD seems to be interesting. But the experiments can be improved to make the paper more convincing. The detailed comments are listed as follows:1. The title (or contribution) of this paper is to design DISTRIBUTED methods for tensor factorization. However, only one short section (Section 3.1), which contains only several sentences, is used to describe the distributed algorithms. More details about the distributed framework should be provided and studied. Actually, the distributed programming models will greatly affect the scalability of machine learning algorithms. More specifically, the authors only mention MapReduce in the paper. However, it is well known that MapReduce is not a good programming model for machine learning algorithms with many iterations. Unfortunately, the algorithms proposed in this paper need many iterations.2. In experiments, the authors only report the results about time per iteration. As mentioned above, time per iteration cannot actually reflect the efficiency of a distributed learning algorithm. The authors should report the whole time for several iterations untill convergence, which will also include the communication time and file reading/writing overhead. 3. In Section 3, the C and B are not defined although we can find them in the supplementary materials.4. The reference style should be consistent.