Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions
Various 3D semantic attributes such as segmentation masks, geometric features, keypoints, and materials can be encoded as per-point probe functions on 3D geometries. Given a collection of related 3D shapes, we consider how to jointly analyze such probe functions over different shapes, and how to discover common latent structures using a neural network -even in the absence of any correspondence information. Our network is trained on point cloud representations of shape geometry and associated semantic functions on that point cloud. These functions express a shared semantic understanding of the shapes but are not coordinated in any way. For example, in a segmentation task, the functions can be indicator functions of arbitrary sets of shape parts, with the particular combination involved not known to the network. Our network is able to produce a small dictionary of basis functions for each shape, a dictionary whose span includes the semantic functions provided for that shape. Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes. We demonstrate the effectiveness of our technique in various segmentation and keypoint selection applications.
Main idea --------- In this work, the authors use a neural network to discover common latent structures in a given collection of related 3D shapes. They formalize the problem as finding shape-dependent dictionaries of basis functions that are meant to capture semantic information shared among all shapes. Interestingly, the aim is to discover such structures even in the absence of any correspondence information.   Strengths - Contributions ------------------------- Introduction is well written and related work seems well described.   1. The model does not require (costly) precomputed basis functions (such as Laplace-Beltrami eigenfunctions) as most approaches.  2. Dictionaries are application-driven, atoms can approximate non-continuous / non-smooth functions.  3. No need to explicitly define canonical bases to synchronize shapes, the neural network becomes the synchronizer. 4. No assumption of a universal dictionary, a data-dependent dictionary allows non-linear distortion of atoms but still preserves consistency.  The results seem to improve upon recent works.  Weaknesses ---------- The main section of the paper (section 3) seems carelessly written. Some obvious weaknesses:  - Algorithm 1 seems more confusing, than clarifying:  a) Shouldn't the gradient step be taken in the direction of the gradient of the loss with respect to Theta?  b) There is no description of the variables, most importantly X and f. It is better for the reader to define them in the algorithm than later in the text.  Otherwise, the algorithm definition seems unnecessary.  - Equation (1) is very unclear:   a) Is the purpose to define a loss function or the optimization problem? It seems that it is mixing both.   b) The optimization variable x is defined to be in R^n. Probably it is meant to be in R^k?   c) The constraints notation (s.t. C(A, x)) is rather unusual.    - It is briefly mentioned that an alternating direction method is used to solve the min-min problem. Which method?  - The constraints in equation (2) are identical to the ones in equation (3). They can be mentioned as such to gain space.  - In section 4.1, line 194, K = 10, presumably refers to the number of atoms in the dictionary, namely it should be a small k?  The same holds for section 4.4, line 285.  - In section 4.1, why is the regularizer coefficient gamma set to zero?  Intuitively, structured sparcity should be particularly helpful in finding keypoint correspondences.  What is the effect on the solution when gamma is larger than zero?   The experimental section of the paper seems well written (with a few exceptions, see above). Nevertheless, the experiments in 4.2 and 4.3 each compare to only one existing work.  In general, I can see the idea of the paper has merit, but the carelessness in the formulations in the main part and the lack of comparisons to other works make me hesitant to accept it as is at NIPS.