In this paper, we diagnose deep neural networks for 3D point cloud processing to explore the utility of different network architectures. We propose a number of hypotheses on the effects of specific network architectures on the representation capacity of DNNs. In order to prove the hypotheses, we design five metrics to diagnose various types of DNNs from the following perspectives, information discarding, information concentration, rotation robustness, adversarial robustness, and neighborhood inconsistency. We conduct comparative studies based on such metrics to verify the hypotheses, which may shed new lights on the architectural design of neural networks. Experiments demonstrated the effectiveness of our method. The code will be released when this paper is accepted. Recently, a series of works use the deep neural network (DNN) for 3D point cloud processing and have achieved superior performance in various 3D tasks. However, traditional studies usually design network architectures based on empiricism. There does not exist a rigorous and quantitative analysis about the utility of specific network architectures for 3D point cloud processing. Exploring and verifying the utility of each specific intermediate-layer architecture from the perspective of a DNN's representation capacity still present significant challenges for state-of-the-art algorithms. In this study, we aim to bridge the gap between the intermediate-layer network architecture and its utility. Therefore, we propose a few hypotheses of the utility of specific network architectures. Table 1 lists the hypotheses to be verified in this study, towards three kinds of utilities, i.e. rotation robustness, adversarial robustness, and neighborhood inconsistency. We design and conduct comparative studies to verify the hypotheses. Finally, we obtain some new insights into the utility of specific network architectures as follows. • The specific architecture in , which uses the local density information to reweight features (Figure 1 (a) ), improves adversarial robustness (Table 1 (a)). • Another specific architecture in , which uses local 3D coordinates' information to reweight features (Figure 1 (b) ), improves rotation robustness (Table 1 (b)). • The specific architecture in (Qi et al., 2017b; Liu et al., 2018) , which extracts multi-scale features (Figure 1 (c) ), improves adversarial robustness and neighborhood consistency (Table 1 (c)). Neighborhood consistency measures whether a DNN assigns similar attention to neighboring points. • The specific architecture in (Jiang et al., 2018) , which encodes the information of different orientations (Figure 1 (d) ), improves rotation robustness (Table 1 (d)) . More specifically, in order to verify the above hypotheses, we design the following five evaluation metrics and conduct a number of comparative experiments to quantify the utility of different network architectures. 1. Information discarding and 2. information concentration: Information discarding measures how much information of an input point cloud is forgotten during the computation of a specific intermediate-layer feature. From the perspective of information propagation, the forward propagation through layers can be regarded as a hierarchical process of discarding input information (Shwartz-Ziv & Tishby, 2017) . Ideally, the DNN is supposed to discard information that is not related to the task. Let us take the task of object classification for example. The information of • Notation: Let xi ∈ R 3 denote the i-th point, i = 1, 2, . . . , n; let N(i) denote a set of K nearest points of xi; let Fi ∈ R d×K denote intermediate-layer features that correspond to neighboring points in N(i), where each column of Fi represents the feature of a specific point in N(i). • Architecture 1, features reweighted by the information of the local density: Architecture 1 focuses on the use of the local density information to reweight features . As shown in Figure 1 (a), for each point xi, Architecture 1 uses the local density w.r.t. neighboring points of xi to compute W H1 ∈ R K , which reweights intermediate-layer features Fi. where diag[W H1 ] transforms the vector W H1 into a diagonal matrix; density(N(i)) is a density vector w.r.t. points in N(i); the M LP is a two-layer perceptron network. • Architecture 2, features reweighted by the information of local coordinates: As shown in Figure 1 ( where the M LP is a single-layer perceptron network. • Architecture 3, multi-scale features: Architecture 3 focuses on the use of multi-scale contextual information (Qi et al., 2017b; Liu et al., 2018) . As illustrated in Figure 1 (c), } denote features that are extracted using contexts of xi at different scales, . Architecture 3 concatenates these multi-scale features to obtain f where where concat indicates the concatenation operator; g(·) is a function for feature extraction (Qi et al., 2017a) . Please see Appendix B for details about this function. • Architecture 4, orientation-aware features: Architecture 4 focuses on the use of orientation information (Jiang et al., 2018) . As illustrated in Figure where Conv oe is a special convolution operator. Please see (Jiang et al., 2018) or Appendix C.4 for details about this operator and the computation of f oe i . In this paper, we have verified a few hypotheses of the utility of four specific network architectures for 3D point cloud processing. Comparative studies are conducted to prove the utility of the specific architectures, including rotation robustness, adversarial robustness, and neighborhood inconsistency. In preliminary experiments, we have verified that Architecture 2 and Architecture 4 mainly improve the rotation robustness; Architecture 1 and Architecture 3 have positive effects on adversarial robustness; Architecture 3 usually alleviates the neighborhood inconsistency. For a better understanding of different versions of DNNs in the next section, we briefly introduce DNNs used in comparative studies, including PointNet++, PointConv, Point2Sequence, and PointSIFT. PointNet++ (Qi et al., 2017b ) is a hierarchical structure composed of a number of set abstraction modules (SA module). For each SA module, a set of points is processed and abstracted to produce a new set with fewer elements. An SA module includes four parts: the Sampling layer, the Grouping layer, the MLP, and the Maxpooling layer. Given a set of N input points, the Sampling layer uses the farthest point sampling algorithm to select a subset of points from the input points, which defines the centroids of local regions, {xi}, i = 1, . . . , N . Then, for each selected point, the Grouping layer constructs a local region by using ball query search to find K neighboring points within a radius r. For each local region N(i) centered at xi, Fi ∈ R d×K denotes the intermediate-layer features that correspond to points in N(i). The MLP transforms Fi into higher dimension features F i ∈ R D×K , where D > d. Finally, the Maxpooling layer encodes F i into a local feature f upper i , which will be fed to the upper SA module. Please see Appendix B for details about the Maxpooling layer.
We diagnose deep neural networks for 3D point cloud processing to explore the utility of different network architectures.  The paper investigates different neural network architectures for 3D point cloud processing and proposes metrics for adversarial robustness, rotational robustness, and neighborhood consistency.