summaries: This paper measures the sensitivity of various point cloud classification networks (like PointNet, PointNet++, DGCNN) to rotation, translation, scale, and local 3D structure. It also proposes ways to measure the 3D smoothness of a network. Based on these analyses, the work provides various insights. One of the main insights being that the DNNs don’t handle rotation very well. Another insight is that training with adversarial perturbations helps. At a higher level, this paper is proposing several ways to measure the sensitivity and smoothness of representations learnt by 3D point cloud networks. This paper presents several ways to measure the sensitivity of point cloud networks to rotation, scale, translation and local structure. The Paper also proposes different metrics to evaluate the smoothness of encoded local 3d structures and the complexity of representations learnt by the network. The analysis is done using several point-based neural networks trained on the ModelNet40 classification task and shapenet part segmentation task. This paper introduces metrics that reflect the representation quality, and properties, acquired by a deep-net that is applied in 3D Point-Cloud (PC) processing tasks. Concretely, the authors introduce six novel metrics that in aggregate capture different types of regional sensitivities e.g., the sensitivity of the DNN in rotations on specific regions/rotation-angles of the input PC. They further explore the extent that modern architectures in standard benchmarks acquire “spatial smoothness” --  do nearby points have ‘similar’ effects/importance for the end task? And finally, they explore the acquired representation complexity of the DNN by adapting a 'multi-order interaction' metric to PC data.    These metrics are being applied on a well-selected subset of modern PC-DNN architectures such PointNet++ or DGCNN when these are trained either for PC-object classification, or part-based object-segmentation. The analysis of the metrics in these tasks/architectures highlights a number of interesting points that oftentimes are very simple/intuitive: e.g., PointNet fails to encode local 3D structures (e.g., compared to PointNet++) — or includes results that are more subtle, but still highly intuitive & rich: e.g., adversarially training a PC-DNN with rotation/translation attacks; will force it to learn more global PC structures compared to standard training; which is intuitive as the local structures, per the new metrics, are those that on average are more susceptible to rotations. This paper studies how to understand the quality of knowledge representations encoded in point cloud DNNs. Based upon the Shapley value, the paper presents six metrics to evaluate the model vulnerability toward different types of input variations. In addition, it presents an analysis regarding the smoothness and multi-order interactions of these attributions, pointing out the restrictions of current point DNNs. The paper makes interesting observations and draws useful conclusions, supported by a range of experimental evaluations.
abstractText: In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the overall model vulnerability into the sensitivity to the rotation, the translation, the scale, and local 3D structures. Besides, we also propose metrics to evaluate the spatial smoothness of encoding 3D structures, and the representation complexity of the DNN. Based on such analysis, experiments expose representation problems with classic DNNs, and explain the utility of the adversarial training. The code will be released when this paper is accepted.
authors.name: Wen Shenb
authors.name: Qihan Ren
authors.name: Dongrui Liu
authors.name: Quanshi Zhanga
references.authors: Marco Ancona Cengiz Oztireli Markus Gross
references.title: Explaining deep neural networks with a polynomial time algorithm for shapley value approximation
references.venue: Proceedings of the 36th International Conference on Machine Learning,
references.authors: Song Bai Xiang Bai Zhichao Zhou Zhaoxiang Zhang Longin Jan Latecki
references.title: Gift: A real-time and scalable 3d shape search engine
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Yizhak Ben-Shabat Michael Lindenbaum Anath Fischer
references.title: 3d point cloud classification and segmentation using 3d modified fisher vector representation for convolutional neural networks
references.venue: arXiv preprint arXiv:1711.08241,
references.authors: Javier Castro Daniel Gómez Juan Tejada
references.title: Polynomial calculation of the shapley value based on sampling
references.venue: Computers & Operations Research,
references.authors: Aditya Chattopadhay Anirban Sarkar Prantik Howlader Vineeth N Balasubramanian
references.title: Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks
references.venue: IEEE Winter Conference on Applications of Computer Vision (WACV),
references.authors: J. Demantké C. Mallet N. David B. Vallet
references.title: Dimensionality Based Scale Selection in 3d LIDAR Point Clouds
references.venue: ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,
references.authors: Alexey Dosovitskiy Thomas Brox
references.title: Inverting visual representations with convolutional networks
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Ruth C Fong Andrea Vedaldi
references.title: Interpretable explanations of black boxes by meaningful perturbation
references.venue: In Proceedings of the IEEE International Conference on Computer Vision,
references.authors: C. Frye Damien de Mijolla T. Begley Laurence Cowton Megan Stanley Ilya Feige
references.title: Shapley explainability on the data manifold
references.venue: In ICLR,
references.authors: Ankit Goyal Hei Law Bowei Liu Alejandro Newell Jia Deng
references.title: Revisiting point cloud shape classification with a simple and effective baseline
references.venue: arXiv preprint arXiv:2106.05304,
references.authors: Michel Grabisch Marc Roubens
references.title: An axiomatic approach to the concept of interaction among players in cooperative games
references.venue: International Journal of game theory,
references.authors: S. Guinard L. Landrieu
references.title: Weakly supervised segmentation-aided classification of urban scenes from 3d lidar point clouds. The International Archives of the Photogrammetry
references.venue: Remote Sensing and Spatial Information Sciences,
references.authors: Qingyong Hu Bo Yang Linhai Xie Stefano Rosa Yulan Guo Zhihua Wang Niki Trigoni Andrew Markham
references.title: Randla-net: Efficient semantic segmentation of large-scale point clouds
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Asako Kanezaki Yasuyuki Matsushita Yoshifumi Nishida
references.title: Rotationnet: Joint object categorization and pose estimation using multiviews from unsupervised viewpoints
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Pieter-Jan Kindermans Kristof T Schütt Maximilian Alber Klaus-Robert Müller Dumitru Erhan Been Kim Sven Dähne
references.title: Learning how to explain neural networks: Patternnet and patternattribution
references.venue: arXiv preprint arXiv:1705.05598,
references.authors: Alexey Kurakin Ian Goodfellow Samy Bengio
references.title: Adversarial examples in the physical world, 2017
references.authors: Loic Landrieu Martin Simonovsky
references.title: Large-scale point cloud semantic segmentation with superpoint graphs
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Yangyan Li Rui Bu Mingchao Sun Wei Wu Xinhan Di Baoquan Chen
references.title: Pointcnn: Convolution on x-transformed points
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Yongcheng Liu Bin Fan Shiming Xiang Chunhong Pan
references.title: Relation-shape convolutional neural network for point cloud analysis
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Scott Lundberg Su-In Lee
references.title: A unified approach to interpreting model predictions
references.venue: arXiv preprint arXiv:1705.07874,
references.authors: Scott M Lundberg Gabriel G Erion Su-In Lee
references.title: Consistent individualized feature attribution for tree ensembles
references.venue: arXiv preprint arXiv:1802.03888,
references.authors: Aravindh Mahendran Andrea Vedaldi
references.title: Understanding deep image representations by inverting them
references.venue: In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition,
references.authors: Jiageng Mao Xiaogang Wang Hongsheng Li
references.title: Interpolated convolutional networks for 3d point cloud understanding
references.venue: In Proceedings of the IEEE/CVF International Conference on Computer Vision,
references.authors: Charles R Qi Hao Su Kaichun Mo Leonidas J Guibas
references.title: Pointnet: Deep learning on point sets for 3d classification and segmentation
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Charles R Qi Or Litany Kaiming He Leonidas J Guibas
references.title: Deep hough voting for 3d object detection in point clouds
references.venue: In Proceedings of the IEEE/CVF International Conference on Computer Vision,
references.authors: Charles Ruizhongtai Qi Li Yi Hao Su Leonidas J Guibas
references.title: Pointnet++: Deep hierarchical feature learning on point sets in a metric space
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Marco Túlio Ribeiro Sameer Singh Carlos Guestrin
references.title: why should I trust you?": Explaining the predictions of any classifier
references.venue: In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining,
references.authors: Ramprasaath R Selvaraju Michael Cogswell Abhishek Das Ramakrishna Vedantam Devi Parikh Dhruv Batra
references.title: Grad-cam: Visual explanations from deep networks via gradient-based localization
references.venue: In Proceedings of the IEEE international conference on computer vision,
references.authors: Lloyd S Shapley
references.title: A value for n-person games
references.venue: Contributions to the Theory of Games,
references.authors: Wen Shen Binbin Zhang Shikun Huang Zhihua Wei Quanshi Zhang
references.title: 3d-rotation-equivariant quaternion neural networks
references.venue: In Computer Vision–ECCV 2020: 16th European Conference,
references.authors: Wen Shen Zhihua Wei Shikun Huang Binbin Zhang Panyue Chen Ping Zhao Quanshi Zhang
references.title: Verifiability and predictability: Interpreting utilities of network architectures for point cloud processing
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Yiru Shen Chen Feng Yaoqing Yang Dong Tian
references.title: Mining point cloud local structures by kernel correlation and graph pooling
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: K Simonyan A Vedaldi A Zisserman
references.title: Deep inside convolutional networks: visualising image classification models and saliency maps
references.venue: arXiv preprint arXiv:1312.6034,
references.authors: Karen Simonyan Andrea Vedaldi Andrew Zisserman
references.title: Deep inside convolutional networks: Visualising image classification models and saliency maps
references.venue: arXiv preprint arXiv:1312.6034,
references.authors: Mukund Sundararajan Amir Najmi
references.title: The many shapley values for model explanation
references.venue: In International Conference on Machine Learning,
references.authors: Mukund Sundararajan Kedar Dhamdhere Ashish Agarwal
references.title: The shapley taylor interaction index
references.venue: In International Conference on Machine Learning,
references.authors: Hugues Thomas Charles R Qi Jean-Emmanuel Deschaud Beatriz Marcotegui François Goulette Leonidas J Guibas
references.title: Kpconv: Flexible and deformable convolution for point clouds
references.venue: In Proceedings of the IEEE International Conference on Computer Vision,
references.authors: Lei Wang Yuchun Huang Yaolin Hou Shenman Zhang Jie Shan
references.title: Graph attention convolution for point cloud semantic segmentation
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Weiyue Wang Ronald Yu Qiangui Huang Ulrich Neumann
references.title: Sgpn: Similarity group proposal network for 3d point cloud instance segmentation
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Yue Wang Justin M Solomon
references.title: Deep closest point: Learning representations for point cloud registration
references.venue: In Proceedings of the IEEE/CVF International Conference on Computer Vision,
references.authors: Yue Wang Yongbin Sun Ziwei Liu Sanjay E Sarma Michael M Bronstein Justin M Solomon
references.title: Dynamic graph cnn for learning on point clouds
references.venue: ACM Transactions on Graphics (TOG),
references.authors: Robert J Weber
references.title: Probabilistic values for games. The Shapley Value
references.venue: Essays in Honor of Lloyd S. Shapley,
references.authors: Wenxuan Wu Zhongang Qi Li Fuxin
references.title: Pointconv: Deep convolutional networks on 3d point clouds
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Zhirong Wu Shuran Song Aditya Khosla Fisher Yu Linguang Zhang Xiaoou Tang Jianxiong Xiao
references.title: 3d shapenets: A deep representation for volumetric shapes
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Guandao Yang Xun Huang Zekun Hao Ming-Yu Liu Serge Belongie Bharath Hariharan
references.title: Pointflow: 3d point cloud generation with continuous normalizing flows
references.venue: In Proceedings of the IEEE/CVF International Conference on Computer Vision,
references.authors: Yaoqing Yang Chen Feng Yiru Shen Dong Tian
references.title: Foldingnet: Point cloud auto-encoder via deep grid deformation
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Li Yi Vladimir G. Kim Duygu Ceylan I-Chao Shen Mengyan Yan Hao Su Cewu Lu Qixing Huang Alla Sheffer Leonidas Guibas
references.title: A scalable active framework for region annotation in 3d shape collections
references.venue: SIGGRAPH Asia,
references.authors: Jason Yosinski Jeff Clune Anh Nguyen Thomas Fuchs Hod Lipson
references.title: Understanding neural networks through deep visualization
references.venue: arXiv preprint arXiv:1506.06579,
references.authors: Lequan Yu Xianzhi Li Chi-Wing Fu Daniel Cohen-Or Pheng-Ann Heng
references.title: Pu-net: Point cloud upsampling network
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Matthew D Zeiler Rob Fergus
references.title: Visualizing and understanding convolutional networks
references.venue: In European Conference on Computer Vision,
references.authors: Hao Zhang Xu Cheng Yiting Chen Quanshi Zhang
references.title: Game-theoretic interactions of different orders
references.venue: arXiv preprint arXiv:2010.14978,
references.authors: Yue Zhao Yuwei Wu Caihua Chen Andrew Lim
references.title: On isometry robustness of deep 3d point cloud models under adversarial attacks
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: Tianhang Zheng Changyou Chen Junsong Yuan Bo Li Kui Ren
references.title: Pointcloud saliency maps
references.venue: In Proceedings of the IEEE/CVF International Conference on Computer Vision,
references.authors: Bolei Zhou Aditya Khosla Agata Lapedriza Aude Oliva Antonio Torralba
references.title: Object detectors emerge in deep scene cnns
references.venue: In ICLR,
references.authors: Bolei Zhou Aditya Khosla Agata Lapedriza Aude Oliva Antonio Torralba
references.title: Learning deep features for discriminative localization
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
sections.heading: 1 Introduction
sections.text: Deep neural networks (DNNs) have exhibited superior performance in various tasks, but the blackbox nature of DNNs hampers the analysis of knowledge representations. Previous studies on explainable AI mainly focused on the following two directions. The first is to explain the knowledge encoded in a DNN, e.g. visualizing patterns encoded by a DNN [24, 36, 50] and estimating the saliency/importance/attribution of input variables w.r.t. the network output [20, 30, 57]. The second is to evaluate the representation power of a DNN, e.g. the generalization and robustness of feature representations.
  This paper focuses on the intersection of the above two directions, i.e. analyzing the quality of knowledge representations of DNNs for 3D point cloud processing. Specifically, we aim to design metrics to illustrate properties of feature representations for different point cloud regions, including various types of regional sensitivities, the spatial smoothness of encoding 3D structures, and the representation complexity of a DNN. These metrics provide new perspectives to diagnose DNNs for 3D point clouds. For example, unlike the image processing relying on color information, the processing of 3D point clouds usually exclusively uses 3D structural information for classification. Therefore, a well-trained DNN for 3D point cloud processing is supposed to just use scale information and 3D structures for inference, and be robust to the rotation and translation.
  Regional sensitivities. We first propose six metrics to disentangle the overall model vulnerability into the regional rotation sensitivity, the regional translation sensitivity, the regional scale sensitivity, and three types of regional structure sensitivity (sensitivity to edges, surfaces, and masses), so that we can use such sensitivity metrics to evaluate the representation quality of a DNN (as Fig. 1 (a-b) shows). Each sensitivity metric (let us take the rotation sensitivity for example) is defined as the vulnerability of the regional attribution when we rotate the 3D point cloud with different angles. The regional attribution is computed as the Shapley value, which is widely used as a unique unbiased estimation of an input variable’s attribution [11, 20, 31, 37] from the perspective of game theory. ∗This work was done when Wen Shen was an intern at Shanghai Jiao Tong University. †Quanshi Zhang is the corresponding author. This study was done under the supervision of Dr. Quanshi Zhang. He is with the John Hopcroft Center and the MoE Key Lab of Artificial Intelligence, AI Institute, at the Shanghai Jiao Tong University, China.
  35th Conference on Neural Information Processing Systems (NeurIPS 2021).
  high low sensitivity sensitivity high rotation sensitivity low translation sensitivity scale sensitivity sensitivity to edges sensitivity to surfaces sensitivity to masses (a)
  high
  low
  sensitivity
  sensitivity
  Based on the sensitivity metrics, we have discovered the following new insights, which have exposed problems with classic DNNs.
  • Insight 1. Rotation robustness is the Achilles’ heel of most DNNs for point cloud processing. The rotation sensitivities of PointNet [26], PointNet++ [28], DGCNN [43], the non-dynamic version of DGCNN (referred to as GCNN in this paper) [43], and PointConv [45] are much higher than other sensitivities (see Fig. 1 (a)).
  • Insight 2. It is usually difficult for DNNs to extract rotation-robust features from 3D points at edges and corners. These points are usually vulnerable to rotation-based attacks.
  • Insight 3. What’s worse, DNNs usually cannot ignore features of such points at edges and corners. Instead, such rotation-sensitive points usually have large regional attributions.
  • Insight 4. PointNet fails to model local 3D structures, because convolutional operations in PointNet encode each point independently. Thus, PointNet has low sensitivity to local 3D structures.
  Spatial smoothness. Beyond the sensitivity metrics, we also evaluate the spatial smoothness of knowledge representations of DNNs. We define the spatial smoothness of knowledge representations as the similarity among the neighboring regions’ attributions to network output. Most widely-used benchmark datasets for point cloud classification [46, 49] only contain objects with simple 3D structures (e.g. a bed mainly composed of a few surfaces in Fig. 1 (c)), in which adjacent regions usually have similar 3D structures. Therefore, most adjacent regions are supposed to have similar attributions to the network output. We also find that the adversarial training increases the spatial smoothness of knowledge representations.
  Representation complexity. Besides regional sensitivities, we further extend the metric of multiorder interaction [53] to evaluate the representation complexity of a DNN, i.e. the maximum complexity of 3D structures that can be encoded in a DNN. The interaction of the m-th order measures the additional benefits brought by collaboration between two point cloud regions i, j under the contexts of other m regions, where m reflects the contextual complexity of the collaboration between regions i and j. High-order interactions usually represent global and complex 3D structures, corresponding to the collaboration between point cloud regions i, j and massive other regions. Loworder interactions usually describe local and simple 3D structures, without being influenced by many contextual regions (see Fig. 1 (d)). Based on such interactions, we have discovered the following new insights.
  • Insight 5. Most DNNs fail to encode high-order interactions. I.e. most DNNs mainly focus on local structures and do not extract many global and complex structures from normal samples.
  • Insight 6. Rotation-sensitive regions usually activate abnormal high-order interactions (global and complex structures). The very limited global structures modeled by the DNN are usually over-fitted to training samples. Thus, when the sample is rotated, the activated global and complex structures usually appear as abnormal patterns.
  • Insight 7. Besides, in terms of model robustness, adversarial training based on adversarial samples w.r.t. using rotations/translations for attack is a typical way to improve the robustness to rotation and translation. Thus, we use the proposed sensitivity metrics to explain the inner mechanism of how and why the adversarially trained DNN is robust to rotation and translation. We find that (1) the adversarial training shifts the attention from the global orientations and positions to the structural information, thereby boosting the sensitivity to local 3D structures. (2) The adversarial training usually increases both the quantity and the complexity of 3D structures (high-order interactions from normal samples) modeled in a DNN, which boosts the robustness to the rotation and translation.
sections.heading: 2 Related work
sections.text: Deep learning on 3D point clouds: Recently, a series of studies directly used DNNs to process 3D point clouds and have achieved promising performance in various 3D tasks [26, 51, 42, 13, 27, 47, 41, 18, 10, 32, 33]. PointNet [26] was the pioneer to use point-wise multi-layer perceptron to process point clouds and aggregate all individual point features into a global feature. To further extract information from local 3D structures, PointNet++ [28] recursively applied PointNet to capture hierarchical 3D structures. Likewise, Relation-Shape CNN [19] and PointCNN [18] also focused on hierarchical structures and improved the ability to extract contextual information. Due to the irregularity of 3D point clouds, some approaches considered the point set as a graph and further defined graph convolutions. DGCNN [43] dynamically constructed local graphs and conducted feature extraction via the EdgeConv operation. SPG [17] built a superpoint graph to process largescale point clouds. GACNet [40] introduced an attention mechanism in graph convolutional networks. KCNet [34] proposed kernel correlation and graph pooling to aggregate contextual information. In addition, many approaches have been proposed to apply convolution operators to the point clouds. PointConv [45] and KPConv [39] utilized nonlinear functions to construct convolution weights from the input 3D coordinates. InterpCNN [23] interpolated features of 3D points to the discrete kernel weights coordinates, so as to implement the discrete convolution operator to point clouds. Instead of designing new architectures, our study focuses on the analysis of representation quality of existing classic DNNs for point cloud processing.
  Visualization or diagnosis of representations: It is intuitive to interpret DNNs by visualizing the feature representations encoded in intermediate layers of DNNs [52, 35, 50, 22, 7], and estimating the pixel-wise saliency/importance/attribution of input variables w.r.t the network output [29, 15, 8, 57, 30, 5, 56]. Similarly, in the 3D domain, PointNet [26] visualized the subset of 3D points (namely the critical subset) that directly affected the network output. Additionally, Zheng et al. [55] specified the critical subset via building a gradient-based saliency map. In comparison, our study proposes to evaluate the representation quality of different point cloud regions.
  Quantitative evaluation of representations: The quantitative evaluation of the representations of DNNs provides a new perspective for explanations. The Shapley value [31] estimates the attribution distribution over all players in a game and has been applied to quantitative evaluation of the representations of DNNs [1, 9, 20]. Based on Shapley values, some studies have investigated the interaction between input variables of a DNN [11, 21, 38, 53]. In comparison, our study aims to illustrate distinctive properties of 3D point cloud processing.
sections.heading: 3 Analyzing feature representations of DNNs for 3D point cloud processing
sections.text: We propose six metrics to measure regional sensitivities to rotation, translation, scale, and three types of local 3D structures, respectively, so as to exhibit the representation property of each specific point cloud region. Beyond these metrics, we can further analyze the representation quality from the following two perspectives, i.e. the spatial smoothness of knowledge representations, and the contextual complexity of 3D structures encoded by a DNN.
  Preliminaries: quantifying the regional attribution using Shapley values. The Shapley value was originally introduced in game theory [31]. Considering a game with multiple players, each player aims to pursue a high award for victory. The Shapley value is widely considered as a unique unbiased approach that allocates the total reward to each player fairly, which satisfies axioms of linearity, nullity, symmetry, and efficiency [44] as the theoretical foundation. Please see our supplementary materials for details.
  Given a point cloud with n regions1, N = {1, 2, · · · , n}, the Shapley value can be used to measure the attribution of each input region. We can consider the prediction process of a DNN as a game v, and each region in a point cloud as a player i. Let 2N def= {S | S ⊆ N} denote all the possible subsets of N . Let xS denote the point cloud only containing regions in S, in which regions in N\S are removed. For a DNN learned for multi-category classification, we use v(S) to denote the network output given the input xS . v(S) is calculated as log p1−p , where p = p(y = y
  truth | xS) denotes the probability of the ground-truth category given xS . Note that the number of input points for each DNN is a fixed value. Therefore, in order to make the DNN successfully handle the point cloud xS without regions in N\S, we reset coordinates of points in regions of N\S to the center of the entire point cloud to remove the information of these points, instead of simply deleting these points [55]. Note that the reason for not simply deleting these points is that the number of input points must be greater than the number of points required for specific layerwise operations of DNNs used in this paper. Specifically, the sampling operation at the first layer of PointNet++ and PointConv requires the minimum number of input points to be greater than 512 points. The k-NN-based grouping [43] operation of DGCNN and GCNN requires the number of input points to be greater than 20 points. However, the computation of Shapley values requires to make inference on a point cloud fragment with a single region i (i.e., x{i}), which may have less than 20 points. Therefore, for such DNNs, we can not simply delete 3D points from the point cloud, in order to allow DNNs to work normally. To this end, for a fair comparison, we choose the alternative way, i.e., placing all points, which are supposed to be masked, to the center of the point cloud. In this way, the numerical attribution of the i-th region to the overall prediction score is estimated by the Shapley value φ(i), as follows.
  φ(i) = ∑
  S⊆N\{i} |S|!(n− |S| − 1)! n! (v(S ∪ {i})− v(S)), (1)
  The computation of Equation (1) is NP-hard. Therefore, we use the sampling-based method in [4] to approximate φ(i).
sections.heading: 3.1 Quantifying the representation sensitivity of a DNN
sections.text: To analyze the quality of knowledge representations of a DNN, we define different types of sensitivities using the above regional attribution φ(i). Specifically, we measure six types of sensitivities, including the rotation sensitivity, translation sensitivity, scale sensitivity, and three types of sensitivity to local 3D structures (edge-like structures, surface-like structures, and mass-like structures). Given an input point cloud x, the Shapley value φ(i) measures the attribution of region i to the network output. The rotation/translation/scale/local-structure sensitivity of this region is quantified as the range of changes of this region’s attribution φ(i) among all potential transformations {T} of the rotation/translation/scale/local 3D structure, as follows.
  ∀ i ∈ N = {1, 2, · · · , n}, ai(x) = 1
  Z (max T φx′=T (x)(i)−min T φx′=T (x)(i)), (2) where Z = ET [ ∑ i∈N |φx′=T (x)(i)|] is computed for normalization. Thus, the average sensitivity to all potential transformations {T} among all input point clouds x ∈ X is formulated as follows.
  sensitivity = Ex∈X [ Ei∈N [ai(x)] ] (3)
  In implementation, six types of sensitivities are computed as follows.
  Rotation sensitivity quantifies the vulnerability of the inference caused by the rotation of 3D point clouds. Given an input point cloud x, we enumerate all rotations θ = [θ1, θ2, θ3]> from the range of [−π4 , π 4 ], i.e. given θ, we sequentially rotate the point cloud around the three axes of the 3D coordinates system, thereby obtaining a set of new point clouds {x′ = Trotation(x|θ)}. Translation sensitivity quantifies the vulnerability of the inference caused by the translation of 3D point clouds. Given an input point cloud x, we enumerate all translations ∆x = [∆x1,∆x2,∆x3]> from the range of [−0.5, 0.5] along the three axes of the 3D coordinates system, thereby obtaining a set of new point clouds {x′ = Ttranslation(x) = x+ ∆x}.
  1There are many ways to segment a 3D point cloud to regions. In our experiments, we first used the farthest point sampling [26] to select n points from each point cloud as centers of n regions. Then, we partitioned the point cloud to n regions by assigning each remaining point to the nearest center point among the selected n center points.
  (b) Ill stration of the spatial smoothness.
  high attrib tion
  lo attrib tion
  not smooth
  smooth
  Scale sensitivity quantifies the vulnerability of the inference caused by the scale of 3D point clouds. Given an input point cloud x, we enumerate all scales α from the range of [0.5, 2], thereby obtaining a set of new point clouds {x′ = Tscale(x) = αx}. Then, we focus on other three sensitivity metrics for three types of local 3D structures, i.e. sensitivity to linearity (edge-like structures), sensitivity to planarity (surface-like structures), and sensitivity to scattering (mass-like structures). According to [12, 6], the significance of a point cloud to be edge-like structures/surface-like structures/mass-like structures is defined as follows.
  linearity = λ1 − λ2 λ1 ; planarity = λ2 − λ3 λ1 ; scattering = λ3 λ1 ; (4)
  where λ1 ≥ λ2 ≥ λ3 denote three eigenvalues of the covariance matrix of a region’s points. Without loss of generality, we take the linearity for example to introduce the way to enumerate a region’s all values of linearity. Given a region, we use the gradient ascent/descent method to modify the 3D coordinates p ∈ R3 of each point in this region, so as to increase/decrease the linearity, i.e. pnew = p + η ∂linearity∂p . As Fig. 2 (a) shows, we enumerate all regions’ linearity at the same time. The enumeration is implemented under the following two constraints. (1) Each value of λ1 needs to be within the range of λ1 ± γ, so as the λ2 and λ3. (2) ‖pfinal − pori‖ ≤ d, where ‖·‖ denotes the L2-norm. In this way, we obtain a set of point clouds with different degrees of local edge-like structures. In real implementation, we set η = 0.001, γ = 0.003, and d = 0.03.
sections.heading: 3.2 Quantifying the spatial smoothness of knowledge representations
sections.text: Besides the analysis of representation sensitivity, we also evaluate the spatial smoothness of feature representations encoded by a DNN. Because most benchmark 3D datasets [46, 49] only contain objects with simple 3D structures (see Fig. 1 (c)), except for special regions (e.g. edges), most adjacent regions of such simple objects usually have similar local 3D structures (e.g. surfaces). Therefore, most adjacent regions are supposed to have similar regional attributions, i.e. high smoothness. In this way, high spatial smoothness indicates reliable feature representations. We quantify the smoothness of the regional attribution between neighboring regions, as follows.
  non-smoothness = Ex∈XETEiEj∈N (i) [ |φx′(i)− φx′(j)|
  Zsmooth
  ∣∣∣ x′=T (x) ] (5)
  whereN (i) denotes a set of nearest point cloud regions (determined by the ball-query search [28]) of region i; Zsmooth = ET [|vx′(N)− vx′(∅)|x′=T (x)] is computed for normalization; vx′(N) denotes the network output given the entire point cloud x′, and vx′(∅) denotes the network output when we mask all regions in x′ by setting all points to the center coordinates of the point cloud.
sections.heading: 3.3 Quantifying the interaction complexity of a DNN
sections.text: In addition, we also quantify the representation complexity of a DNN, i.e. the complexity of 3D structures encoded by a DNN. To this end, the 3D structures encoded by a DNN is represented using the interaction between different 3D point cloud regions. Here, input regions of a DNN do not work individually, but collaborate with each other to construct a specific 3D structure for inference. Zhang et al. [53] defined the multi-order interactions between two input variables. Given two input regions i and j, the interaction of the m-th order measures the additional attribution brought by collaborations
  between i and j under the context of m regions. I(m)(i, j) = ES⊆N\{i,j},|S|=m [ v(S ∪ {i, j})− v(S ∪ {i})− v(S ∪ {j}) + v(S) ] , (6)
  I(m)(i, j) > 0 indicates that the presence of region j increases the attribution of region i, φ(i). I(m)(i, j) < 0 indicates that the presence of region j decreases the value of φ(i). I(m)(i, j) ≈ 0 indicates that region j and region i are almost independent of each other. Please see our supplemental materials for details about the meaning of multi-order interaction and the linearity, nullity, commutativity, symmetry, and efficiency axioms of the multi-order interaction.
  Here, we could consider the order m as the number of contextual regions involved in the computation of interactions between region i and region j. For example, as Fig. 2 (b) shows, regions i and j (in red), and other m regions (in blue) work together to construct a 3D structure to classify the earphone.
  High-order interactions measure the effects of global collaborations among massive regions, i.e. representing complex and large-scale 3D structures. Low-order interactions measure the effects of collaborations between a few regions, i.e. usually representing simple and small-scale 3D structures. Then, we use the following metric to quantify the average strength of the m-th order interactions as the significance of the m-order complex 3D structures.
  I(m) = Ex∈X [∣∣Ei,j [I(m)x (i, j)]∣∣]. (7)
  If the I(m) of a low order is significantly larger than that of a high order, then the representation complexity of the DNN is limited to representing simple and local 3D structures.
sections.heading: 4 Comparative studies
sections.text: In this section, we conducted comparative studies to analyze properties of different point cloud regions of different DNNs. Ideally, a well-trained DNN for 3D point cloud processing was supposed to be robust to the rotation and translation, and the DNN was supposed to mainly use the scale and local 3D structures for inference. Besides, considering the 3D structures of objects in benchmark 3D datasets [46, 49] were usually simple, most adjacent regions in an object had continuous and similar 3D structures. Therefore, a well-trained DNN was supposed to have similar regional attributions among these neighboring regions. We also analyzed how complex is the 3D structure that can be encoded by a classic DNN.
  We used our method to analyze five classic DNNs for 3D point cloud processing, including the PointNet [26], the PointNet++ [28], the DGCNN [43], the non-dynamic version of DGCNN (i.e. GCNN), and the PointConv [45]. All DNNs were learned based on the ModelNet10 dataset [46] and the ShapeNet part2 dataset [49]. We followed [26] to only use 1024 points of each point cloud to train all DNNs. Each point cloud was partitioned to n = 32 regions1 for the computation of all metrics.
  Note that all DNNs used in our paper were well-trained. The testing accuracy was shown in Table 1.We also reported the testing accuracy of other DNNs trained on the ModelNet10 dataset, including RotationNet [14], 3DmFV-Net [3], KCNet [34], and GIFT [2]. These four DNNs obtained the 1st, 4th, 7th, and 10th testing accuracy according the ModelNet10 Benchmark Leaderboard3.
  In particular, the previous study [54] has discovered that people could use rotations to attack DNNs for 3D point cloud processing. Therefore, we used the adversarial training to learn a GCNN4 w.r.t. the attacks based on rotations and translations of point clouds, so as to improve the GCNN’s robustness to the rotation and translation. We extended the method in [16] to generate such adversarial examples to attack the GCNN. Please see our supplemental materials for details. The objective function
  2We only used part of the ShapeNet part dataset due to the time limitation. Please see our supplemental materials for details.
  3Data from https://modelnet.cs.princeton.edu/. 4In tables, adv-GCNN denoted the adversarially trained GCNN.
  was minw Ex [ maxT Loss(x′ = T (x), ytruth;w) ] , where w denoted the parameter of the GCNN. Comparative studies have revealed the following effects of the adversarial training (using rotations and translations for attack, instead of using perturbations) on knowledge representations. The adversarial training shifted the attention of DNN from orientations and positions to structural information (including global structures in normal samples, see Fig. 5), thereby increasing the sensitivity to 3D structures (see Table 2 and Fig. 3 (a)). The adversarial training also decreased the correlation between the regional sensitivity and the regional attribution (see Table 3), and increased the spatial smoothness of knowledge representations (see Table 4).
  Comparative study 1, explaining the regional sensitivity of DNNs. Table 2 shows six types of regional sensitivities of six DNNs learned on the ModelNet10 dataset and the ShapeNet part dataset. In terms of translation sensitivity, PointNet was relatively sensitive to the translation, because PointNet extracted features from global coordinates of points, thereby being sensitive to the translation. In contrast, PointNet++ and PointConv were robust to translation, because these two DNNs encoded relative coordinates. Particularly, PointConv did not use global coordinates during the inference process, thereby yielding the lowest translation sensitivity (see the point cloud with the darkest blue in the second row of Fig. 3 (a)). In terms of scale sensitivity, PointNet++ was relatively sensitive to the scale, because PointNet++ encoded features of neighborhood with fixed scales. Besides, because adversarial training forced the GCNN to remove attention from rotation-sensitive features and translation-sensitive features, the adversarially trained GCNN paid more attention to structural information. Therefore, compared with the original GCNN, the adversarially trained GCNN had higher sensitivity to local 3D structures. Furthermore, we also obtained the following conclusions.
  • Rotation robustness was the Achilles’ heel of classic DNNs for 3D point cloud processing. As Table 2 and Fig. 3 (a) show, all DNNs were sensitive to rotations except for the adversarially trained GCNN.
  • PointNet failed to encode local 3D structures. In terms of sensitivity to local 3D structures, PointNet was the least sensitive DNN (see Table 2). It was because convolution operations of the PointNet encoded the information of each point independently, i.e. the PointNet did not encode the information of neighboring points/regions. This conclusion is also verified by the phenomenon that PointNet had darker blue point clouds than other DNNs (see the last three rows of Fig. 3 (a)).
  • Most DNNs usually failed to extract rotation-robust features from 3D points at edges and corners. Given a point cloud x, we selected the most rotation-sensitive region i∗ (shown as black boxes in Fig. 3 (b)). We rotated the point cloud x to the orientations that maximized and minimized the
  边缘结构敏感度
  平⾯结构敏感度
  团状结构敏感度
  attribution of region i∗, i.e. θ1 = arg maxθ φx′(i∗) and θ2 = arg minθ φx′(i∗). Fig. 3 (b) visualizes regional attributions of point clouds rotated by θ1 and θ2. We found that rotation-sensitive regions were usually distributed on the edges and corners, which verified our conclusion.
  • Most DNNs usually could not ignore features of rotation-sensitive points at edges and corners. Table 3 shows that the Pearson correlation coefficient [25] between each region’s average strength of attribution over different rotations (i.e. Eθ[|φx′(i)|], s.t. x′ = Trotation(x|θ)) and this region’s rotation sensitivity ai(x) is much larger than Pearson correlation coefficients between the attribution and other sensitivities. This means that rotation-sensitive regions (which were usually distributed on the edges and corners) usually had large regional attributions, which verified our conclusion. Besides, Table 3 also shows that the adversarial training reduced the correlation between the regional sensitivity and the regional attribution. Fig. 4 (a) visualizes the positive correlation between the regional rotation sensitivity and the regional attribution of the PointNet.
  Comparative study 2, explaining the spatial smoothness of knowledge representations. Table 4 shows the non-smoothness of knowledge representations encoded by different DNNs. Note that the adversarially trained GCNN was significantly biased to the knife category. In other words, the
  adversarially trained GCNN classified the empty input as the knife category with a high confidence (please see our supplementary materials for details). Therefore, to enable fair comparisons, we also reported the spatial non-smoothness of the adversarially trained GCNN on all other categories except the knife. We discovered that without considering the biased knife category, adversarial training increased the spatial smoothness of knowledge representations. Fig 4 (b) also verified this conclusion.
  Comparative study 3, explaining the interaction complexity of DNNs. Fig. 5 shows multi-order interactions of different DNNs. From this figure, we discovered the following new insights.
  • Most DNNs failed to encode high-order interactions (i.e. global and large-scale 3D structures). As Fig. 5 (a) shows, no matter given normal samples or adversarial samples5, classic DNNs encoded extremely low-order interactions. This indicated that most DNNs did not extract complex and large-scale 3D structures from normal samples.
  • The adversarial training (using rotations and translations for attack, instead of using perturbations) increased the effects of extremely high-order interactions. As Fig. 5 (a) shows, the adversarially trained GCNN encoded extremely low-order interactions and extremely high-order interactions (i.e. the global structures of objects) from normal samples. Because the GCNN was forced to sophisticatedly select relatively complex rotation-robust features among all potential features.
  • Rotation sensitivity of regions with out-of-distribution high-order interactions (i.e. abnormal complex and large-scale 3D structures) were larger than the average sensitivity over all regions. In fact, there are two types of high-order interactions. Unlike the above two insights describing whether or not a DNN encoded high-order interactions on normal samples, this insight mainly focused on abnormal out-of-distribution high-order interactions in a DNN. When a point cloud was attacked by adversarial rotations5, the adversarial rotation5 would generate out-of-distribution high-order interactions (i.e. abnormal complex and large-scale 3D structures) to attack the DNN. We measured interactions between the most rotation-sensitive region i∗ and its neighbors j ∈ N (i∗) among all input point clouds, i.e. Ex∈X [ |Ej∈N (i∗)[I (m) x (i∗, j)]| ] . As Fig. 5 (b) shows, rotation-sensitive regions usually encoded relatively more high-order interactions than the overall interaction distribtuion over all regions. This indicates that compared to most other regions, rotation-sensitive regions paid more attention to high-order interactions, although low-order interactions of rotation-sensitive regions increased too.
  Broad applicability of the proposed metrics. We also conducted experiments on an AutoEncoder for 3D point cloud reconstruction task and rotation-invariant DNNs for 3D point cloud classification, in order to further demonstrate the broad applicability of the proposed metrics.
  For the AutoEncoder for 3D point cloud reconstruction task, we used all layers before the fullyconnected layers in PointNet [26] as the encoder and used the decoder in the FoldingNet [48] as the decoder. We took v(S) = (z(N)−z(∅))
  >(z(S)−z(∅)) ||z(N)−z(∅)||2 as the reward score in Equation (1), where
  z(S) denotes the output vector of the encoder given the input point cloud consisting of regions in S. This reward score measures the utility of the encoder output z(S) given regions in S along the direction of z(N) − z(∅). The AutoEncoder was learned based on the ModelNet10 dataset. We measured the rotation sensitivity (0.099±0.043), the translation sensitivity (0.152±0.086), and the scale sensitivity (0.045±0.042) of the AutoEncoder. Compared with results in Table 2, we found that (1) the AutoEncoder for reconstruction was also sensitive to rotation; (2) the AutoEncoder for 3D
  5Here, we used rotations for attack, instead of using perturbations.
  point cloud reconstruction was more sensitive to translation than the DNN for classification; (3) the AutoEncoder for reconstruction was sensitive to the scale change.
  Besides, we followed [32] to revise PointNet++ and DGCNN to be rotation-invariant DNNs. We measured the rotation sensitivity of the rotation-invariant PointNet++ (0.002±0.001) and DGCNN (0.010±0.004). Compared with results in Table 2, we found that rotation-invariant DNNs were much more robust to rotations than traditional DNNs (even the adversarially-trained DNN).
  Effects of data augmentation on sensitivities. We conducted comparative studies to explore the effects of data augmentation on sensitivities, including translation augmentation, scale augmentation, rotation augmentation around the y-axis, and rotation augmentation around a random axis. We conducted experiments on PointNet and DGCNN. To explore the effects of each of the above augmentation on sensitivities, we learned two versions of each network architecture, i.e. one network with the specific augmentation and the other network without the specific augmentation. Please see our supplementary materials for more details about different versions of each network architecture. All DNNs were learned based on the ModelNet10 dataset. Table 5 shows that rotation/translation/scale augmentation decreased the rotation/translation/scale sensitivity of a DNN.
  Besides, we compared the effects of rotation augmentation around a random axis and the effects of adversarial training on the rotation sensitivity. We conducted experiments on DGCNN and GCNN. We trained two versions of DGCNN and GCNN, one with rotation augmentation around a random axis (DGCNN, 0.052 ±0.021; GCNN, 0.048±0.024) and one with the adversarial training using rotations for attack (DGCNN, 0.036±0.015; GCNN, 0.028±0.016). We found that compared with the rotation augmentation, the adversarial training w.r.t. rotation-based attacks had a greater impact on the rotation sensitivity.
sections.heading: 5 Conclusion
sections.text: In this paper, we have measured six types of regional sensitivities, the spatial smoothness, and the contextual complexity of feature representations encoded by DNNs. Comparative studies have discovered several new insights into classic DNNs for 3D point cloud processing. We have found that most DNNs were extremely sensitive to the rotation. Rotation-sensitive regions were usually distributed on the edges and corners of objects, had large attributions to network output, and usually paid more attention to abnormal large-scale structures. In addition, most DNNs failed to extract complex and large-scale 3D structures from normal samples, while the adversarial training could encourage the DNN to extract global structures from normal samples.
  Acknowledgments and Disclosure of Funding
  This work is partially supported by the National Nature Science Foundation of China (No. 61906120, U19B2043), Shanghai Natural Science Fundation (21JC1403800,21ZR1434600), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102). This work is also partially supported by Huawei Technologies Inc.
title: Interpreting Representation Quality of DNNs for 3D Point Cloud Processing