Deep learning applied to the reconstruction of 3D shapes has seen growing interest. A popular approach to 3D reconstruction and generation in recent years has been the CNN decoder-encoder model often applied in voxel space. However this often scales very poorly with the resolution limiting the effectiveness of these models. Several sophisticated alternatives for decoding to 3D shapes have been proposed typically relying on alternative deep learning architectures. We show however in this work that standard benchmarks in 3D reconstruction can be tackled with a surprisingly simple approach: a linear decoder obtained by principal component analysis on the signed distance transform (SDF) of the surface. This approach allows easily scaling to larger resolutions. We show in multiple experiments it is competitive with state of the art methods and also allows the decoder to be fine-tuned on the target task using a loss designed for SDF transforms, obtaining further gains.
This paper studies the problem of learning the feature representation for predicting the 3D shape of objects, from a single image or a point cloud. The proposed approach performs PCA on the SDF field. And then the transformed feature map is learned and used as input to task-specific decoders for 3D shape prediction. The authors claims that this approach trains faster and is easier to scale, while showing competitive performance compared to state-of-the-art methods.
I am leaning towards Weak Reject. The paper is generally easy to read, but with some details missing. And I found the discussion of the results to be insufficient. I think it can be an above-threshold paper if questions are addressed during rebuttal.
Being able to easily scale to higher resolutions is claimed to be one of the main advantages, but I am not convinced that this is useful under this setting. If I understand this correctly, the number of eigenvectors k is fixed, and projecting the SDF field to this space would remove the higher frequency components of the shape. So wouldn't the number of eigenvectors be the bottleneck in representational precision, not the resolution of the output space?
What is the chosen k (number of eigenvectors)? It says k was "chosen to capture 99.5% of the variance within the dataset", but I could not find how exactly it was chosen and what value of k was used (I apologize if I missed).
Also, I think the PCA is category-specific (page 4, section 4.1). Is k dependent on the category or is it the same across all categories? Some of the other methods (if not all) used for comparison are not category-specific, so if this is true, I think the comparison may not be entirely fair and it should be made clearer.
I think the writing could be polished as well, some minor typos:
Page 2:  3rd and 4th paragraphs: continous
Page 2: anlaysis, enlightning
Page3: under eigenSDF: reprsentation
Page 4: section 4.1: refered, signficant
Page 5: “seciton”
Page 7: tranform
Page 3, Section 3.2: Is N the number of training examples and M the resolution?
Figure 2 not referred in the main text.