models afﬁnities between pixels in an image, and images
naturally lend themselves to multiscale or pyramid-like
representations and algorithms.
Our algorithm is inspired by two observations: 1) if
Ais bistochastic (the rows and columns of Asum to 1)
then the eigenvectors of the Laplacian Aare equal to
the eigenvectors of the Laplacian of A2, and 2) because
of the scale-similar nature of images, the eigenvectors
of a “downsampled” version of Ain which every other
pixel has been removed should be similar to the eigenvectors of A. Let us deﬁne pixel decimate (A), which
takes an afﬁnity matrix Aand returns the set of indices
of rows/columns in Acorresponding to a decimated
version of the image from which Awas constructed. That
is, ifi=pixel decimate (A), thenA[i;i]is a decimated
matrix in which alternating rows and columns of the imagehave been removed. Computing the eigenvectors of
A[i;i]works poorly, as decimation disconnects pixels in
the afﬁnity matrix, but the eigenvectors of the decimated
We propose a unified approach for bottom-up hierarchical image segmentation
and object proposal generation for recognition, called Multiscale Combinatorial
Grouping (MCG). For this purpose, we first develop a fast normalized cuts
algorithm. We then propose a high-performance hierarchical segmenter that makes
effective use of multiscale information. Finally, we propose a grouping
strategy that combines our multiscale regions into highly-accurate object
proposals by exploring efficiently their combinatorial space. We also present
Single-scale Combinatorial Grouping (SCG), a faster version of MCG that
produces competitive proposals in under five second per image. We conduct an
extensive and comprehensive empirical validation on the BSDS500, SegVOC12, SBD,
and COCO datasets, showing that MCG produces state-of-the-art contours,
hierarchical regions, and object proposals.
Jordi Pont-Tuset Pablo Arbelaez Jonathan T. Barron Ferran Marques Jitendra Malik