FPNN: Field Probing Neural Networks for 3D Data
Building discriminative representations for 3D data has been an important task in computer graphics and computer vision research. Convolutional Neural Networks (CNNs) have shown to operate on 2D images with great success for a variety of tasks. Lifting convolution operators to 3D (3DCNNs) seems like a plausible and promising next step. Unfortunately, the computational complexity of 3D CNNs grows cubically with respect to voxel resolution. Moreover, since most 3D geometry representations are boundary based, occupied regions do not increase proportionately with the size of the discretization, resulting in wasted computation. In this work, we represent 3D spaces as volumetric fields, and propose a novel design that employs field probing filters to efficiently extract features from them. Each field probing filter is a set of probing points -sensors that perceive the space. Our learning algorithm optimizes not only the weights associated with the probing points, but also their locations, which deforms the shape of the probing filters and adaptively distributes them in 3D space. The optimized probing points sense the 3D space "intelligently", rather than operating blindly over the entire domain. We show that field probing is significantly more efficient than 3DCNNs, while providing state-of-the-art performance, on classification tasks for 3D object recognition benchmark datasets.
This paper focuses on the problem of object classification based on 3D data that one could acquire with the use of some sort of range sensor. The basic idea is that for 3D data one would ideally like to have 3D filters in a convolutional neural network. However, 3D filters are quite expensive to be optimized and used, and they are pretty wasteful since the majority of the input will probably be zeros (only near the object surface there are non-zero values). As a note here, the input is a 3D rectangle split into a 3D grid, where the original object point cloud is converted in a simplified, binarized form (you can think of it as building the surface of an object using LEGO blocks).  As such the paper proposes a field probing layer which replaces the convolutional layers with a combination of three "sub-layers": a) a sensor layer, b) a dot-product layer and c) a gaussian layer. The sensor layer learns to place sampling points/probes on the 3D rectangle space, which would be equivalent to learning the sampling/convolutional locations in a ConvNet. The gaussian layer implements a gaussian function that smoothens the sampling points from the sensor layer. And the dot-product is a standard inner product layer to compute the inner product between the sampling layer and some learnt weights. Interestingly, the weights are not shared in the field probing layer, meaning that in different locations different "probing filters" are learnt. A strong positive point is that the proposed approach can learn complex interaction between surface points, even if they are not neighboring ones. The reason is that the sensor layers are not constrained to place the probes within a confined space.  
The paper is quite interesting, although not entirely novel or delivering impressive results. The list of positive and negative points is the following.  Positive points ------------------- + The problem is interesting and an approach that could deliver some good results would definitely have some impact. Indeed, relying on filters with cubic complexity for 3DCNN is unnecessarily intense, since most of the input space is composed of zeros. Learning more "surface-focused" filters make much more sense. + Modelling long-range surface dependencies is sensible, especially when scaling up to datasets with hundreds of categories. + The presentation of the paper is very good and the method is very clear.  Negative points ------------------- - In its current form a field probing can be viewed as some sort of sparse tensor encoding. As such it is not clear what is the novelty as compared to having a sparse encoding of the input space and the weight filters. This is also kind of supported by figure 5, where indeed the proposed method has a straight line (the same one would expect by a sparse encoding), whereas a 3D convolutional network naturally feature a cubic complexity. One difference I can identify is the fact that long-range dependencies can be learnt, namely that the the probing locations are learnt instead of just defined. Is that a difference and is it the only main difference? - It is not clear how can you learn meaningful long-range dependencies within such an unconstrained setting. With normal convolutional filters there exists a very strong constraint, namely the inputs should be nearby. In your case how can you make sure the network does not get stuck in a poor local optimum? - The fact that the weights in the field probing layers are not shared means that they are tied to the particular locations of the input. How generalizable is that? When you are presented with a random point cloud for which you cannot know the rotation as compared to the ground truth, how could these filters cope with the variance? - Extending on the point, how do you perform the rotation changes in Table 2? Do you apply the same, fixed rotation to all objects or do you rotate randomly per test object? If it is the former, does the experiment bring any insight, as I assume that the classification should be more or less similar for any random (but common across examples) viewpoint. If it is the latter, what is the difference between R and R_15? - Also, since the weights are not shared, does the network generalize for objects of different sizes, since in that case one would naturally need to increase the granularity so that to be able to maintain the appopriate level of detail for the recognition? If this cannot be guaranteed, doesn't this go against the point of having a more efficient network so that one can increase the grid resolution on demand?  Overall I believe the paper is quite interesting and although by itself does not excel, it could serve as a potential inspiration to other researchers.