Parallel Double Greedy Submodular Maximization
Many machine learning problems can be reduced to the maximization of submodular functions. Although well understood in the serial setting, the parallel maximization of submodular functions remains an open area of research with recent results [1] only addressing monotone functions. The optimal algorithm for maximizing the more general class of non-monotone submodular functions was introduced by Buchbinder et al. [2] and follows a strongly serial double-greedy logic and program analysis. In this work, we propose two methods to parallelize the double-greedy algorithm. The first, coordination-free approach emphasizes speed at the cost of a weaker approximation guarantee. The second, concurrency control approach guarantees a tight 1/2-approximation, at the quantifiable cost of additional coordination and reduced parallelism. As a consequence we explore the tradeoff space between guaranteed performance and objective optimality. We implement and evaluate both algorithms on multi-core hardware and billion edge graphs, demonstrating both the scalability and tradeoffs of each approach.
Many submodular maximization could be optimized with decent approximation ratio by greedy-like algorithms. However, due to the inherently serial nature of the algorithms, they could not be easily parallelized. Leveraging the transaction processing model from database system, this paper presents two approaches to parallelizing the double greedy algorithm for unconstrained submodular maximization. In particular, the double greedy algorithm maintains two sets A and B where one can only add elements to A or removed elements from B. The authors consider each such operation an transaction just like the one in database system. The problem is then reduced to how to apply these transactions in parallel. The first approach (CF-2g) this paper proposes is a hogwild-style coordination-free algorithm where each thread runs locally with a possibly stale version of A and B. Due to the submodularity, the marginal improvement computed within each thread is always an upper bound of the true one. The authors also analyze the approximation ratio of CF-2g.The second approach (CC-2g) is a concurrency-control algorithm which guarantees the serializability and thus it shares the same approximation ratio as the serial double-greedy algorithm. It works by maintaining the upper and lower bound of A and B, which can be used to locally compute the upper bound and lower bound of the threshold in the serial algorithm. This "often" enables each thread to locally decide to add the element to A or remove it from B. And uncertain transactions are recomputed serially by the server. Figure 2 is a nice illustration of this process. As a consequence CF-2g promotes efficiency and CC-2g promotes solution quality. At the end, the authors conduct comprehensive experiments to evaluate the proposed algorithms. But seems like CC-2g algorithm is comparable with CF-2g in terms of speedup, and it even has more speedup than CF-2g in some cases (e.g. Fig. 3c) which is counter-intuitive. Is that due to some specific problem structure? I think the authors should give more discussions to make it more clear. Also, as pointed out in the last section, the delays might hurt the performance in distributed environment. But so far it's unclear empirically.