We address the problem of learning to discover 3D parts for objects in unseen categories. Being able to learn the geometry prior of parts and transfer this prior to unseen categories pose fundamental challenges on data-driven shape segmentation approaches. Formulated as a contextual bandit problem, we propose a learningbased iterative grouping framework which learns a grouping policy to progressively group small part proposals into bigger ones in a bottom-up fashion. At the core of our approach is to restrict the local context for extracting part-level features, which encourages the generalizability to novel categories. On a recently proposed large-scale fine-grained 3D part dataset, PartNet, we demonstrate that our method can transfer knowledge of parts learned from 3 training categories to 21 unseen testing categories without seeing any annotated samples. Quantitative comparisons against four strong shape segmentation baselines shows that we achieve the state-of-the-art performance.
This paper proposes a method for part segmentation in object pointclouds. The method is to (1) break the object into superpixel-like subparts (without semantic meaning yet), then (2) score pairs of parts on their mergeability, (3) greedily merge the best pair, and repeat. The scoring has a unary component (called a "purity" module), and a pairwise component (called a "rectification" module); the unary component determines if the joined pointcloud of two sub-parts appears part-like, and the pairwise component determines if the features of the two sub-parts appear compatible. These components are implemented as pointnets/MLPs. Finally there is a termination module, which sigmoid-scores part pairs on whether they should actually merge (and the algorithm continue), or not (and we stop). The purity and termination modules are trained supervised, to mimic intersection-like and mergeability scores, and the rectification module with a "reward" which is another mergeability score (coming from GT and the purity module).
The method is interesting for being (1) iterative, and (2) driven by purely local cues. The iterative approach, with small networks doing the work, is a nice relief from the giant-network baselines (such as PartNet-InsSeg) that take the entire pointcloud as input and produce all instance segmentations directly. Also, whereas most works try to maximize the amount of contextual input to the learning modules, this work makes the (almost certainly correct) observation that the smaller the contextual input, the smaller the risk for overfitting. This is a bit like making the approach "convolutional", in the sense that the same few parameters are used repeatedly over space (and in this case, also repeated over scale). The design of the local modules makes sense, although I would prefer they be called unary/pairwise instead of purity/rectification, and the RL training procedure looks reasonable also.
I am not totally clear on how the termination module actually comes into play. From the name, it sounds like this network would output 1 when the algorithm should terminate, but in its usage, it seems to output 1 when the best-scored pair should be merged. So then, does the algorithm terminate when this module decides to NOT merge the best-scored pair? This sounds like it bears great risk of early stopping. I would appreciate some clarification on this.
The abstract says that locality "guarantees the generalizability to novel categories". This is an overstatement, since "guarantees" implies some theoretical proof, and also since the paper's own results (in Table 1 and 3) indicate that cross-category generalization is far from addressed, and depends partly on the categories used in training (shown in Table 2). 
I assume that this method has (or at least can have) far fewer parameters than the baselines, since the components never need to learn broad contextual priors. Can the authors clarify and elaborate on this please? If you can show that your method has far fewer parameters than the baselines, it would improve the paper I think.
Can the authors please provide some statistics on the earliest stage of the method, where superpixel-like parts are proposed? How many proposals, and how many pairs does this make, and how slowly do the main modules proceed through these pairs? 
Is there a missing step that makes the part selection non-random? It seems like many of the pairs can be rejected outright early on, such as ones whose centroids exceed some distance threshold in 3D.