SubmodBoxes: Near-Optimal Search for a Set of Diverse Object Proposals
This paper formulates the search for a set of bounding boxes (as needed in object proposal generation) as a monotone submodular maximization problem over the space of all possible bounding boxes in an image. Since the number of possible bounding boxes in an image is very large O(#pixels 2 ), even a single linear scan to perform the greedy augmentation for submodular maximization is intractable. Thus, we formulate the greedy augmentation step as a Branch-and-Bound scheme. In order to speed up repeated application of B&B, we propose a novel generalization of Minoux's 'lazy greedy' algorithm to the B&B tree. Theoretically, our proposed formulation provides a new understanding to the problem, and contains classic heuristic approaches such as Sliding Window+Non-Maximal Suppression (NMS) and and Efficient Subwindow Search (ESS) as special cases. Empirically, we show that our approach leads to a state-of-art performance on object proposal generation via a novel diversity measure.
SUMMARYThe paper introduces a principled approach to generate a diverse set of relevant object proposals. This is formulated as optimizing a function F defined on sets of bounding boxes. This function is defined as a sum of a relevance term (which encourages individual bounding boxes to be likely to contain an object) and a diversity term. For optimization they commit to a greedy approach which iteratively adds the most promising bounding box, until the desired number of proposals was generated. The greedy procedure was motivated by existing theoretical guarantees on the obtained solution when F is submodular and monotone. Even under this greedy procedure, the computational cost would be prohibitive when using the naive approach of scanning all possible bounding boxes in order to find the best one. To this end, a Branch and Bound (B&B) procedure is employed. Furthermore, the efficiency of the B&B procedure is improved by sharing information between successive steps of the greedy procedure, generalizing ideas from Minoux's Lazy Greedy procedure.COMMENTSOn the positive side, the paper introduces an extensible framework for object proposal generation, which is shown to encompass some previous heuristics like NMS and the one used in the ESS [22] method. The branch and bound procedure is interesting and makes it feasible to (greedily) search for the solution despite the very large number of candidate bounding boxes considered at each step. This involved constructing an upper bound for the marginal gain of F given a set of candidate bounding boxes. One technical novelty consists of generalizing Minoux's Lazy Greedy procedure in order to leverage information from the B&B tree constructed in the previous step of the greedy procedure. This procedure was shown to perform well in practice. Furthermore, the paper is clearly written and well organized.There appears to be an issue with the monotonicity and submodularity proofs presented in the supplementary material. In brief, this is related to the fact that these concepts are defined on functions over sets (not over lists), and the theoretical guarantees mentioned refer to this situation. The proposed method is defined in terms of a list (lines 179-181), but the diversity function should be defined on the corresponding unordered sets (not lists) in order for the theoretical guarantees to be available. However, the definition in eqs. 7a, 7b can lead to different values of the diversity function D, for two lists which differ only in the order of the proposals (e.g. for different orderings, the same proposal might be attached to different reference bounding boxes, leading to different values when summing up the contributions from all proposals) -- therefore the elements of the underlying set cannot be considered in an arbitrary order when evaluating D, so the proof has to be adjusted. How do these aspects influence the theoretical claims regarding the performance of the greedy procedure?The results in figure 4 show a comparison between the proposed method and variants of Edge Boxes and Selective Search. It would be best to compare to additional state-of-the-art methods, such as MCG and CPMC (which are present in table 1). The results in table 1 are obtained under the regime of a small number of proposals (200 per image, but methods like MCG and CPMC can be tuned for as many as thousands of proposals if not more), which is not frequently used in practice. For such reasons, the impact of the proposed method is not clear from an empirical perspective.The extension of the greedy procedure was shown in fig. 6 to lead to evaluating significantly fewer nodes of the B&B graph. It would be interesting to also show the effect in terms of execution time.Minor comments: there is a double caption for fig. 3.