Beyond the Birkhoff Polytope: Convex Relaxations for Vector Permutation Problems
The Birkhoff polytope (the convex hull of the set of permutation matrices), which is represented using Θ(n 2 ) variables and constraints, is frequently invoked in formulating relaxations of optimization problems over permutations. Using a recent construction of Goemans [1], we show that when optimizing over the convex hull of the permutation vectors (the permutahedron), we can reduce the number of variables and constraints to Θ(n log n) in theory and Θ(n log 2 n) in practice. We modify the recent convex formulation of the 2-SUM problem introduced by Fogel et al. [2] to use this polytope, and demonstrate how we can attain results of similar quality in significantly less computational time for large n. To our knowledge, this is the first usage of Goemans' compact formulation of the permutahedron in a convex optimization problem. We also introduce a simpler regularization scheme for this convex formulation of the 2-SUM problem that yields good empirical results.
This paper studies the seriation problem where n objects with known characteristics must be ordered in a sequence such that a prescribed similarity measure of the characteristics decreases with the distance of the objects in the sequence. The noiseless seriation problem is formalized as the recovery of an R-matrix (whose entries are non-increasing as we move away from the diagonal horizontally or vertically) from a similarity matrix that has the structure of a pre-R-matrix (i.e., an R-matrix with permuted rows and columns). In the noisy seriation problem the similarity matrix is only approximately equal to a pre-R-matrix, and thus no exact R-matrix can be recovered.The noiseless seriation problem can be formulated as a 2-SUM optimization problem, which minimizes a convex quadratic form over all permutations of the integer vector (1,2,...,n). As the 2-SUM problem is NP-hard, this paper suggests to solve a convex relaxation of the 2-SUM problem over the convex hull of all permutations. It is argued that such a relaxation can also offer high-quality solutions for the noisy seriation problem. This approach requires a heuristic to recover a valid permutation from the solution of the relaxed problem.The convex hull of all permutations of (1,2,...,n) can be seen as a projection of the Birkhoff polytope of permutation matrices, which has a representation in terms of n^2 variables and constraints. This approach was adopted in reference [2]. The present paper suggests to use a more efficient sorting network representation of the convex hull of all permutations, which was discovered in [1] and only involves n*log(n) variables and constraints. While the noiseless seriation problem can be solved exactly and efficiently with a spectral method, the proposed relaxation method (as well as the relaxation method from [2]) provide good solutions for the noisy problem and when partial prior information about the sequence of the n elements is available (e.g., when some elements satisfy known precedence relations).The paper addresses an important topic with interesting applications in machine learning. The idea to use more efficient descriptions of the permutahedron is a good one, and the numerical results clearly demonstrate the merits of this approach. On the theoretical side, the contributions seem less convincing. It appears that this paper essentially combines an idea from [2] to solve the seriation problem by seeking convex relaxations of a related 2-SUM problem and from [3] to represent the permutahedron using a sorting network. Once these two ideas are combined, everything else is more or less straightforward. This is maybe the biggest weakness of the paper.I did not spot any mathematical errors or inaccuracies.The paper is generally well written and well structured. I just have some minor comments about the exposition:- l128: The sentence "Theorem 2.3 can be used to show that this construction can be used" sounds awkward.- l169: It would be useful to remind the reader that the Laplacian matrix is always positive semidefinite because it is symmetrical and diagonally dominant.- p4: It sounds as if the NP-hardness of the 2-SUM problem is the reason to study convex relaxations of (5). However, it is stated that (5) is efficiently and exactly solvable by the spectral method. This sounds like a contradiction. Of course, there is no contradiction, because the seriation problem is a special case of the 2-SUM problem where the input matrix is a pre-R-matrix, and this special case is tractable. It would be good to make this clearer and to motivate the formulations (7) and (9) by the desire to incorporate prior information and to account for noisy data (in which case the spectral method may no longer be applicable or display an inferior performance). It would be useful to work out these arguments more clearly. I had to read this several times until I (hopefully) understood the logic.- l199: I missed the definition of Pi_I. Is this the permutation matrix corresponding to the trivial permutation pi_I? So it is the identity matrix?- Figure 2(left): I am confused by the meaning of the outer circle and how it relates to the ellipsoidal level curves.- l291: "The asymptotically sorting network" sounds awkward.- l334: typo: "abd" -> "and".- Table 1: The caption states that solver times are averaged over 10 runs. However, the performance measures seem also to be averaged. Please clarify.