A Non-convex One-Pass Framework for Generalized Factorization Machine and Rank-One Matrix Sensing
We develop an efficient alternating framework for learning a generalized version of Factorization Machine (gFM) on steaming data with provable guarantees. When the instances are sampled from d dimensional random Gaussian vectors and the target second order coefficient matrix in gFM is of rank k, our algorithm converges linearly, achieves O( ) recovery error after retrieving O(k 3 d log(1/ )) training instances, consumes O(kd) memory in one-pass of dataset and only requires matrixvector product operations in each iteration. The key ingredient of our framework is a construction of an estimation sequence endowed with a so-called Conditionally Independent RIP condition (CI-RIP). As special cases of gFM, our framework can be applied to symmetric or asymmetric rank-one matrix sensing problems, such as inductive matrix completion and phase retrieval.
This paper proposes a new algorithm for training a variant of Factorization Machines (FMs) (second-order polynomial parameterized by a low-rank matrix).  Pros:  - simple, efficient algorithm - novel convergence/recovery analysis *assuming* Gaussian-distributed data  Con: - no experimental validation at all  Note that the model considered in this paper is actually not the original FMs but a variant (some authors call it "polynomial networks"; see below). The authors might want to update the title and main body of their paper to reflect this. 
Major comments --------------  * An obvious major issue with this paper is the lack of experiments. How does   the algorithm compare to gradient-based local search algorithms? I would be curious to see if it works better on i) Gaussian distributed data and ii) real data. Even if the results turn out to be similar, perhaps the authors can find some advantages to their algorithm such as, e.g., robustness to initialization.  * Technically, the model considered in this paper is a *variant* of FMs in two ways  1) The model considered uses the quadratic form x^T M x = \sum_{i=1}^d \sum_{j=1}^d M_{i,j} x_i x_j while FMs use \sum_{i=1}^{d-1} \sum_{j=i+1}^d M_{i,j} x_i x_j. Recent works [1, 2] have called this variant "polynomial network" so the authors might want to use this name instead of "factorization machine".  If the proposed algorithm and analysis trivially extends to the original FMs, the authors might want to add a few words about this as well.  [1] https://arxiv.org/abs/1410.1141  [2] http://jmlr.org/proceedings/papers/v48/blondel16.html  2) The original FMs factor M as V V^T, hence M is p.s.d. It would be interesting to briefly discuss this point. In particular, it is not clear whether a p.s.d. constraint is needed or whether it can hurt performance.  * Theorem 1 assumes that the data is Gaussian distributed. This   should be stated clearly in the abstract and introduction too.  Detailed comments -----------------  * Lines 80-81:  > When d is much larger than k, convex programming on the trace norm or nuclear norm of M∗ becomes intractable since M ∗ is of d × d.  This claim is wrong. Modern solvers for nuclear norm penalized minimization have no problem scaling to large d provided that the data is sparse. The authors should make a weaker claim or a more precise statement. For example, "although the estimation of M* can be formulated as convex programming based on the nuclear norm, a more popular strategy in practice is to ...".  * Lines 113-116:  You could also use \hat{x}_i^T M \hat{x}_i (i.e., use \hat{x}_i both on the left and right).  * How difficult is it to extend the proposed algorithm to other losses than the   squared loss?  * Line 123:   Due to the name "one-pass", I assume the instances are sampled without replacement?  * Algorithm 1:   H_3 seems to be a vector so it would be less confusing to not capitalize it.  * The precise objective function minimized in this paper does not seem to be   defined anywhere.  Typos -----  * Line 207: holds true -> hold true