Back

PointCNN: Convolution On X-Transformed Points

Yangyan LiRui BuMingchao SunWei WuXinhan DiBaoquan Chen

article2018NeurIPS2,996 citations

Proposes PointCNN, a framework that generalizes standard convolution to irregular 3D point clouds by learning an $\mathcal{X}$-transformation matrix that simultaneously weights local features and permutes points into a canonical order.

Listen

Point clouds from 3D sensors are irregular and unordered, so standard convolution operators that work well on grid data such as images lose shape information and become sensitive to point ordering. This limits the direct use of proven CNN techniques for tasks such as object classification and scene segmentation that rely on 3D sensing.

The paper introduces PointCNN, a framework that generalizes CNNs to point clouds. Its central operator, X-Conv, learns a K-by-K transformation matrix from local point coordinates with a multilayer perceptron, then uses that matrix to weight and reorder neighboring features before a conventional convolution is applied. The operator is stacked hierarchically with down-sampling and optional dilation, producing networks that aggregate information from increasing receptive fields while remaining computationally practical.

Evaluations on ModelNet40, ScanNet, ShapeNet Parts, S3DIS and additional sketch and image-derived sets show PointCNN reaching or exceeding the accuracy of prior point-cloud networks, with overall accuracies of 92.5 percent on pre-aligned ModelNet40 and 86.14 percent part-averaged IoU on ShapeNet Parts. Ablation tests confirm that removing the learned transformation drops accuracy by roughly 1.5 points even when model depth and parameter count are matched. The same architecture also performs competitively on 2D sketches and MNIST points, though a larger gap remains on CIFAR10 points that lack clear shape structure. Runtime measurements indicate low parameter counts and inference times on the order of 0.012 seconds per batch for classification.

These results indicate that a learned, neighborhood-specific reordering step can restore the spatial-correlation benefits of convolution without forcing point clouds into dense grids. The approach therefore offers a practical route to hierarchical feature learning on raw 3D data for robotics, autonomous driving and indoor mapping applications.

Further gains may come from tighter integration with image-based CNNs on paired data and from refined sampling strategies that handle highly non-uniform point distributions. The current X-transformations achieve only partial permutation invariance, so additional theoretical analysis of the operator inside deep stacks would strengthen confidence in scaling the method. Code is released to support such extensions.

HuaweiPeking UniversityShandong Universityinput featurespoint cloudrepresentation learningPointCNN modelpoint cloud representationgeometric deep learning
arXiv: 1801.07791yangyanli/PointCNN
Cover for PointCNN: Convolution On X-Transformed Points

Abstract

We present a simple and general framework for feature learning from point clouds. The key to the success of CNNs is the convolution operator that is capable of leveraging spatially-local correlation in data represented densely in grids (e.g. images). However, point clouds are irregular and unordered, thus directly convolving kernels against features associated with the points, will result in desertion of shape information and variance to point ordering. To address these problems, we propose to learn an X\mathcal{X}-transformation from the input points, to simultaneously promote two causes. The first is the weighting of the input features associated with the points, and the second is the permutation of the points into a latent and potentially canonical order. Element-wise product and sum operations of the typical convolution operator are subsequently applied on the X\mathcal{X}-transformed features. The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus we call it PointCNN. Experiments show that PointCNN achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.

Topics

input featurespoint cloudrepresentation learningshape informationfeature spacecontrol pointcorrelation coefficientfast fourier transforms

Keywords

PointCNN modelpoint cloud representationgeometric deep learninghierarchical convolutionSpatial Transformer Networkssemantic segmentationconvolution operatorShapeNet Parts

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 PointCNN
  • 3.1 Hierarchical Convolution
  • 3.2 𝒳\mathcal{X}-Conv Operator
  • 3.3 PointCNN Architectures
  • 4 Experiments
  • 4.1 Classification and Segmentation Results
  • 4.2 Ablation Experiments and Visualizations
  • 5 Conclusion
  • References
  • 1 𝒳\mathcal{X}-Conv Details
  • 2 Dataset Details
  • 3 PointCNN Model Zoo
  • 4 Detailed Segmentation Results

Knowls

  1. Knowl 1 — The X-Conv Operator for Point Cloud Feature Learning

    algorithm

    The X\mathcal{X}-Conv operator is the foundational building block of PointCNN. It generalizes standard grid convolution to unordered point clouds by learning an input-dependent transformation matrix X∈RK×K\mathcal{X} \in \mathbb{R}^{K \times K} that simultaneously weights and permutes local point features before convolving them with standard trainable filters K∈RK×(C1+Cδ)×C2\mathbf{K} \in \mathbb{R}^{K \times (C_1 + C_\delta) \times C_2}.

    Given a representative point p∈RDimp \in \mathbb{R}^{\text{Dim}}, its KK nearest neighbors P=(p1,p2,…,pK)T∈RK×Dim\mathbf{P} = (p_1, p_2, \dots, p_K)^T \in \mathbb{R}^{K \times \text{Dim}}, and their associated input features F=(f1,f2,…,fK)T∈RK×C1\mathbf{F} = (f_1, f_2, \dots, f_K)^T \in \mathbb{R}^{K \times C_1}, the X\mathcal{X}-Conv operator is computed as:

    Fp=X-Conv(K,p,P,F)=Conv(K,MLP(P−p)×[MLPδ(P−p),F])\mathbf{F}_p = \mathcal{X}\text{-Conv}(\mathbf{K}, p, \mathbf{P}, \mathbf{F}) = \text{Conv}\left(\mathbf{K}, \text{MLP}(\mathbf{P} - p) \times [\text{MLP}_\delta(\mathbf{P} - p), \mathbf{F}]\right)

    where [⋅,⋅][\cdot, \cdot] denotes feature concatenation along channels, MLPδ(⋅)\text{MLP}_\delta(\cdot) individually projects local coordinates into a CδC_\delta-dimensional feature space, MLP(⋅)\text{MLP}(\cdot) estimates the K×KK \times K permutation and weighting matrix X\mathcal{X}, and Conv(⋅,⋅)\text{Conv}(\cdot, \cdot) applies standard convolution.

    Input: Trainable kernel K\mathbf{K}, representative point p∈RDimp \in \mathbb{R}^{\text{Dim}}, neighbor points P=(p1,…,pK)T∈RK×Dim\mathbf{P} = (p_1, \dots, p_K)^T \in \mathbb{R}^{K \times \text{Dim}}, neighbor features F=(f1,…,fK)T∈RK×C1\mathbf{F} = (f_1, \dots, f_K)^T \in \mathbb{R}^{K \times C_1}
    Output: Output feature vector Fp∈RC2\mathbf{F}_p \in \mathbb{R}^{C_2} at representative point pp
    P′←P−p\mathbf{P}' \leftarrow \mathbf{P} - p
    Fδ←MLPδ(P′)\mathbf{F}_\delta \leftarrow \text{MLP}_\delta(\mathbf{P}')
    F∗←[Fδ,F]\mathbf{F}_* \leftarrow [\mathbf{F}_\delta, \mathbf{F}]
    X←MLP(P′)\mathcal{X} \leftarrow \text{MLP}(\mathbf{P}')
    FX←X×F∗\mathbf{F}_\mathcal{X} \leftarrow \mathcal{X} \times \mathbf{F}_*
    Fp←Conv(K,FX)\mathbf{F}_p \leftarrow \text{Conv}(\mathbf{K}, \mathbf{F}_\mathcal{X})
    return Fp\mathbf{F}_p

  2. Knowl 2 — Parameterization and Computational Reduction of the X-Transformation Matrix

    model/method

    In X\mathcal{X}-Conv, learning the X\mathcal{X}-transformation matrix X∈RK×K\mathcal{X} \in \mathbb{R}^{K \times K} from neighbor coordinates P′∈RK×3\mathbf{P}' \in \mathbb{R}^{K \times 3} using fully connected layers across flattened vectors of size K×KK \times K requires O(K4)\mathcal{O}(K^4) parameters, which induces prohibitive memory usage and severe overfitting risks.

    To reduce complexity to O(K3)\mathcal{O}(K^3) parameters, PointCNN parameterizes MLP(P′)\text{MLP}(\mathbf{P}') using a combination of fully connected and depthwise convolutional layers:

    FC(3K,K×K)→ELU→BN→Reshape(K×K,K×K)→DC(K×K,K×K)→ELU→BN→Reshape(K×K,K×K)→DC(K×K,K×K)→BN→Reshape(K×K,K×K)\text{FC}(3K, K \times K) \to \text{ELU} \to \text{BN} \to \text{Reshape}(K \times K, K \times K) \to \text{DC}(K \times K, K \times K) \to \text{ELU} \to \text{BN} \to \text{Reshape}(K \times K, K \times K) \to \text{DC}(K \times K, K \times K) \to \text{BN} \to \text{Reshape}(K \times K, K \times K)

    where DC(R×C,C×F)\text{DC}(R \times C, C \times F) represents a depthwise convolution applying FF distinct filters to each of the CC columns of an R×CR \times C input matrix, requiring only R×C×FR \times C \times F trainable parameters instead of (R⋅C)×(C⋅F)(R \cdot C) \times (C \cdot F) parameters.

  3. Knowl 3 — Coordinate Feature Lifting and Separable Convolution in X-Conv

    model/method

    The X\mathcal{X}-Conv layer uses specialized implementations for both coordinate lifting (MLPδ\text{MLP}_\delta) and feature convolution (Conv\text{Conv}):

    1. Local Coordinate Lifting (MLPδ\text{MLP}_\delta): Local relative coordinates P′=P−p\mathbf{P}' = \mathbf{P} - p are lifted into a CδC_\delta-dimensional space using a point-wise multilayer perceptron with Exponential Linear Units (ELU) and Batch Normalization (BN): FC(3,Cδ)→ELU→BN→FC(Cδ,Cδ)→ELU→BN\text{FC}(3, C_\delta) \to \text{ELU} \to \text{BN} \to \text{FC}(C_\delta, C_\delta) \to \text{ELU} \to \text{BN} where CδC_\delta is configured as Cδ=C1/4C_\delta = C_1 / 4.

    2. Separable Convolution: A standard convolution between kernel K\mathbf{K} and FX∈RK×(C1+Cδ)\mathbf{F}_\mathcal{X} \in \mathbb{R}^{K \times (C_1 + C_\delta)} would require K×(C1+Cδ)×C2K \times (C_1 + C_\delta) \times C_2 parameters. PointCNN implements Conv(K,FX)\text{Conv}(\mathbf{K}, \mathbf{F}_\mathcal{X}) as a depthwise separable convolution with depth multiplier DM=⌈C2/(C1+Cδ)⌉DM = \lceil C_2 / (C_1 + C_\delta) \rceil, reducing parameter count to: K×(C1+Cδ)×DM+(C1+Cδ)×DM×C2K \times (C_1 + C_\delta) \times DM + (C_1 + C_\delta) \times DM \times C_2

  4. Knowl 4 — Hierarchical PointCNN Architecture and Dilated Point Convolution

    model/method

    PointCNN hierarchically aggregates point cloud representations across layers, shrinking the number of representative points from N1N_1 to N2N_2 (N2<N1N_2 < N_1) while expanding the channel depth from C1C_1 to C2C_2 (C2>C1C_2 > C_1).

    • Representative Point Selection: Representative points {p2,i}\{p_{2,i}\} are selected via random downsampling for classification tasks and farthest point sampling (FPS) for segmentation tasks to ensure spatial uniformity.
    • Dilated Point Convolution: The receptive field of a representative point is defined by (K×D)/N(K \times D) / N, where NN is the number of points in the preceding layer, KK is the number of neighbors convolved, and DD is the dilation rate. Instead of taking the immediate KK nearest neighbors, PointCNN uniformly samples KK points from the K×DK \times D nearest neighbor set. This expands the receptive field from K/NK/N to (K×D)/N(K \times D)/N without increasing the neighbor count KK or kernel parameters.

  5. Knowl 5 — PointCNN Architectures for Classification and Semantic Segmentation

    model/method

    PointCNN structures deep networks for classification and dense point labeling tasks using X\mathcal{X}-Conv layers:

    • Classification Network: Uses hierarchical X\mathcal{X}-Conv layers with dilated convolutions. Rather than reducing the point count to 1 at the final convolutional stage, multiple representative points (e.g., N=128N=128) are retained to ensure thorough training of upper layers. Each remaining point has a receptive field covering the entire shape (D≥2D \ge 2). Their individual feature predictions through shared fully connected (FC) layers are averaged right before the softmax layer.
    • Segmentation Network (Conv-DeConv / U-Net): High-resolution point-wise output is achieved using a symmetrical encoder-decoder architecture where both Conv (downsampling) and DeConv (upsampling) layers use the X\mathcal{X}-Conv operator. Higher-resolution coordinates and features from earlier Conv layers are forwarded via skip-connections to DeConv layers. To incorporate subvolume supervision, global representative point coordinates in the final X\mathcal{X}-Conv layer are mapped to Cg=C2/4C_g = C_2 / 4 dimensions via MLPg(p):FC(3,Cg)→ELU→BN→FC(Cg,Cg)→ELU→BN\text{MLP}_g(p): \text{FC}(3, C_g) \to \text{ELU} \to \text{BN} \to \text{FC}(C_g, C_g) \to \text{ELU} \to \text{BN} and concatenated into the feature stream.

  6. Knowl 6 — Dynamic Point Subsampling and Shuffling Augmentation

    model/method

    To train the X\mathcal{X}-transformation matrix to generalize across arbitrary point permutations and local density variations, fixed neighborhoods and ordering must not be reused across epochs.

    For a network configured to process NN points at test time, PointCNN generates training samples by randomly drawing N′∼N(N,(N/8)2)N' \sim \mathcal{N}(N, (N/8)^2) points from each point cloud and randomly shuffling their order before each training batch. This forces the X\mathcal{X}-transformation to dynamically learn canonical permutations and feature weightings on varying neighbor configurations.

  7. Knowl 7 — Empirical Classification Performance on 3D Benchmarks

    data/table

    PointCNN was evaluated on ModelNet40 (1024 input points) under both Pre-aligned (no random horizontal rotation) and Unaligned (random horizontal rotation applied to both train and test sets) settings, as well as on ScanNet (17-category object instance classification from 1024 points).

    ModelNet40 ScanNet
    Pre-aligned Unaligned Unaligned
    Method mA (%) OA (%) mA (%) OA (%) mA (%) OA (%)
    Flex-Convolution - 90.2 - - - -
    KCNet - 91.0 - - - -
    Kd-Net 88.5 90.6 - - - -
    SO-Net - 90.7 - - - -
    3DmFV-Net - 91.4 - - - -
    PCNN - 92.3 - - - -
    PointNet - - 86.2 89.2 - -
    PointNet++ - - - 90.7 - 76.1
    SpecGCN - - - 91.5 - -
    SpiderCNN - - - - - -
    DGCNN - - 90.2 92.2 - -
    PointCNN 88.8 92.5 88.1 92.2 55.7 79.7

    PointCNN achieves top classification accuracy on ModelNet40 (92.5% pre-aligned OA, 92.2% unaligned OA) and outperforms PointNet++ on ScanNet object classification by 3.6% OA (79.7% vs. 76.1%).

  8. Knowl 8 — Empirical 3D Semantic and Part Segmentation Performance

    data/table

    PointCNN was evaluated on three 3D segmentation benchmarks: ShapeNet Parts (evaluated by part-averaged IoU, pIoU %, and mean per-class part-averaged IoU, mpIoU %), Stanford Large-Scale 3D Indoor Spaces (S3DIS, evaluated by overall accuracy OA %, micro-averaged accuracy mAcc %, and micro-averaged IoU mIoU %), and ScanNet (voxel OA %).

    ShapeNet Parts S3DIS ScanNet
    Method pIoU (%) mpIoU (%) OA (%) mAcc (%) mIoU (%) OA (%)
    PointNet 83.7 80.4 78.5 66.2 47.6 73.9
    PointNet++ 85.1 81.9 - - - 84.5
    RSNet 84.9 81.4 - 66.45 56.47 -
    DGCNN 85.1 82.3 - - 56.1 -
    SSCN 85.98 83.3 - - - -
    SPGraph - - 85.5 73.0 62.1 -
    SGPN 85.8 82.8 - - 50.37 -
    PointCNN 86.14 84.6 88.14 75.61 65.39 85.1

    PointCNN surpasses prior state-of-the-art segmentation frameworks across all three benchmarks, achieving 86.14% pIoU on ShapeNet Parts, 65.39% mIoU on S3DIS, and 85.1% voxel OA on ScanNet.

  9. Knowl 9 — Ablation of X-Transformation and Feature Canonization Analysis

    empirical result

    To evaluate the specific contribution of the learned X\mathcal{X}-transformation, PointCNN was compared against three control models on ModelNet40 classification where the X\mathcal{X}-transformation was omitted and standard convolution applied directly on F∗\mathbf{F}_*:

    Model Core Layers # Parameters Accuracy (%)
    PointCNN X-Conv×4\mathcal{X}\text{-Conv} \times 4 0.60M 92.2
    PointCNN w/o X\mathcal{X} Conv×4\text{Conv} \times 4 0.54M 90.7
    PointCNN w/o X\mathcal{X}-W (wider) Conv×4\text{Conv} \times 4 0.63M 90.8
    PointCNN w/o X\mathcal{X}-D (deeper) Conv×5\text{Conv} \times 5 0.61M 90.7

    Wider and deeper variants without X\mathcal{X} match the parameter count and depth of PointCNN but achieve only 90.7%–90.8% accuracy, demonstrating that the performance gain is attributable to X\mathcal{X}-Conv rather than added parameters or depth.

    Furthermore, nearest-center cluster classification on intermediate features across diverse point input orderings yields the following discriminability scores:

    • Untransformed lifted features from PointCNN without X\mathcal{X} (FoF_o): 76.83%
    • Lifted features before X\mathcal{X}-transformation (F∗F_*): 89.29%
    • Features transformed by X\mathcal{X} (FXF_\mathcal{X}): 94.72%

    This quantitatively verifies that the learned X\mathcal{X}-transformation projects unordered, varying point features into a canonical representation.

  10. Knowl 10 — Evaluation on 2D Sketch and Non-Grid Image Classifications

    empirical result

    PointCNN was tested on 2D non-grid representations including sketch classification (TU-Berlin and Quick Draw) and point-sampled image datasets (MNIST and CIFAR10):

    Method TU-Berlin (%) Quick Draw (%) MNIST (%) CIFAR10 (%)
    LeNet / AlexNet 68.60 - 99.20 84.07
    Network in Network - - 99.53 91.20
    PointNet++ 66.53 51.58 99.49 10.03
    PointCNN 70.57 59.13 99.54 80.22

    On sketch data (represented as 1D strokes in 2D space), PointCNN outperforms generic image CNNs (AlexNet: 68.60% vs. PointCNN: 70.57%) and PointNet++ (51.58% vs. PointCNN: 59.13% on Quick Draw). On CIFAR10 converted to 512 point clouds with RGB features (where geometric shape is absent), PointCNN achieves 80.22% while PointNet++ completely fails (10.03%), demonstrating PointCNN's capability to leverage spatial correlation in feature space.

Coverage note — None was omitted; all key architectural components, algorithmic steps, hyperparameter details, training strategies, and empirical benchmark/ablation results are covered.

References

  1. 1.Martín Abadi and et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.
  2. 2.Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. 3d semantic parsing of large-scale indoor spaces. In CVPR, pages 1534–1543, 2016.
  3. 3.Matan Atzmon, Haggai Maron, and Yaron Lipman. Point convolutional neural networks by extension operators. ACM Trans. Graph., 37(4):71:1–71:12, July 2018.
  4. 4.Yizhak Ben-Shabat, Michael Lindenbaum, and Anath Fischer. 3d point cloud classification and segmentation using 3d modified fisher vector representation for convolutional neural networks. arXiv preprint arXiv:1711.08241, 2018.
  5. 5.Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017.
  6. 6.François Chollet. Xception: Deep learning with depthwise separable convolutions. arXiv preprint arXiv:1610.02357, 2016.
  7. 7.Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). In ICLR, 2016.
  8. 8.Rodrigo Santa Cruz, Basura Fernando, Anoop Cherian, and Stephen Gould. Deeppermnet: Visual permutation learning. In CVPR, July 2017.
  9. 9.Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017.
  10. 10.Sander Dieleman, Jeffrey De Fauw, and Koray Kavukcuoglu. Exploiting cyclic symmetry in convolutional neural networks. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, pages 1889–1898. JMLR.org, 2016.
  11. 11.Mathias Eitz, James Hays, and Marc Alexa. How do humans sketch objects? ToG, 31(4):44:1–44:10, 2012.
  12. 12.Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. arXiv preprint arXiv:1711.10275, 2017.
  13. 13.Benjamin Graham and Laurens van der Maaten. Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307, 2017.
  14. 14.Fabian Groh, Patrick Wieschollek, and Hendrik P. A. Lensch. Flex-convolution (deep learning beyond grid-worlds). arXiv preprint arXiv:1803.07289, 2018.
  15. 15.David Ha and Douglas Eck. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477, 2017.
  16. 16.Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. Transforming auto-encoders. In International Conference on Artificial Neural Networks, pages 44–51. Springer, 2011.
  17. 17.Binh-Son Hua, Minh-Khoi Tran, and Sai-Kit Yeung. Point-wise convolutional neural network. In CVPR, 2018.
  18. 18.Qiangui Huang, Weiyue Wang, and Ulrich Neumann. Recurrent slice networks for 3d segmentation on point clouds. In CVPR, 2018.
  19. 19.Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448–456, 2015.
  20. 20.Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. In Advances in Neural Information Processing Systems, pages 2017–2025, 2015.
  21. 21.Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2014.
  22. 22.Roman Klokov and Victor Lempitsky. Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In ICCV, 2017.
  23. 23.Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097–1105, 2012.
  24. 24.Loïc Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. CoRR, abs/1711.09869, 2017.
  25. 25.Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436–444, 2015.
  26. 26.Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  27. 27.Jiaxin Li, Ben M. Chen, and Gim Hee Lee. So-net: Self-organizing network for point cloud analysis. In CVPR, 2018.
  28. 28.Yangyan Li, Sören Pirk, Hao Su, Charles R Qi, and Leonidas J Guibas. Fpnn: Field probing neural networks for 3d data. In NIPS, pages 307–315, 2016.
  29. 29.Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. In ICLR, 2014.
  30. 30.Haggai Maron, Meirav Galun, Noam Aigerman, Miri Trope, Nadav Dym, Ersin Yumer, Vladimir G. Kim, and Yaron Lipman. Convolutional neural networks on surfaces via seamless toric covers. ACM Trans. Graph., 36(4):71:1–71:10, July 2017.
  31. 31.Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In CVPR, July 2017.
  32. 32.Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learning deconvolution network for semantic segmentation. In ICCV, ICCV ’15, pages 1520–1528, Washington, DC, USA, 2015. IEEE Computer Society.
  33. 33.Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, pages 77–85, July 2017.
  34. 34.Charles R. Qi, Hao Su, Matthias Nießner, Angela Dai, Mengyuan Yan, and Leonidas J. Guibas. Volumetric and multi-view cnns for object classification on 3d data. In CVPR, pages 5648–5656, 2016.
  35. 35.Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NIPS, pages 5105–5114, 2017.
  36. 36.Siamak Ravanbakhsh, Jeff Schneider, and Barnabas Poczos. Deep learning with sets and point clouds. arXiv preprint arXiv:1611.04500, 2016.
  37. 37.Gernot Riegler, Ali Osman Ulusoys, and Andreas Geiger. Octnet: Learning deep 3d representations at high resolutions. In CVPR, 2017.
  38. 38.Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, MICCAI, pages 234–241, Cham, 2015. Springer International Publishing.
  39. 39.David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning internal representations by error propagation. In David E. Rumelhart, James L. McClelland, and CORPORATE PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, pages 318–362. MIT Press, Cambridge, MA, USA, 1986.
  40. 40.Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. Dynamic routing between capsules. In NIPS, pages 3859–3869, 2017.
  41. 41.Tianjia Shao, Yin Yang, Yanlin Weng, Qiming Hou, and Kun Zhou. H-CNN: spatial hashing based CNN for 3d shape analysis. arXiv preprint arXiv:1803.11385, 2018.
  42. 42.Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In CVPR, 2018.
  43. 43.Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz. Splatnet: Sparse lattice networks for point cloud processing. In CVPR, 2018.
  44. 44.Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou. Tangent convolutions for dense prediction in 3d. In CVPR, 2018.
  45. 45.Lyne P. Tchapmi, Christopher B. Choy, Iro Armeni, JunYoung Gwak, and Silvio Savarese. Segcloud: Semantic segmentation of 3d point clouds. In 3DV, 2017.
  46. 46.Chu Wang, Babak Samari, and Kaleem Siddiqi. Local spectral graph convolution for point set feature learning. arXiv preprint arXiv:1803.05827, 2018.
  47. 47.Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-cnn: Octree-based convolutional neural networks for 3d shape analysis. ACM Trans. Graph., 36(4):72:1–72:11, July 2017.
  48. 48.Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun. Deep parametric continuous convolutional neural networks. In CVPR, 2018.
  49. 49.Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann. SGPN: similarity group proposal network for 3d point cloud instance segmentation. In CVPR, 2018.
  50. 50.Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:1801.07829, 2018.
  51. 51.Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. Deep points consolidation. ToG, 34(6):176:1–176:13, October 2015.
  52. 52.Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, pages 1912–1920, 2015.
  53. 53.Yifan Xu, Tianqi Fan, Mingye Xu, Long Zeng, and Yu Qiao. Spidercnn: Deep learning on point sets with parameterized convolutional filters. arXiv preprint arXiv:1803.11527, 2018.
  54. 54.Li Yi, Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ToG, 35(6):210:1–210:12, November 2016.
  55. 55.Li Yi, Hao Su, Xingwen Guo, and Leonidas Guibas. Syncspeccnn: Synchronized spectral cnn for 3d shape segmentation. In CVPR, pages 6584–6592, July 2017.
  56. 56.Li Yi, Hao Su, Lin Shao, Manolis Savva, Haibin Huang, Yang Zhou, Benjamin Graham, Martin Engelcke, Roman Klokov, Victor Lempitsky, et al. Large-scale 3d shape reconstruction and segmentation from shapenet core55. arXiv preprint arXiv:1710.06104, 2017.
  57. 57.Qian Yu, Yongxin Yang, Feng Liu, Yi-Zhe Song, Tao Xiang, and Timothy M. Hospedales. Sketch-a-net: A deep neural network that beats humans. IJCV, 122(3):411–425, May 2017.
  58. 58.Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R. Salakhutdinov, and Alexander J Smola. Deep sets. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, NIPS, pages 3394–3404, 2017.

Citation

MLA
Li, Y., et al. “PointCNN: Convolution On X-Transformed Points”. Advances in Neural Information Processing Systems, vol. 31, 2018, pp. 820–30, https://proceedings.neurips.cc/paper_files/paper/2018/file/f5f8590cd58a54e94377e6ae2eded4d9-Paper.pdf.
APA
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., & Chen, B. (2018). PointCNN: Convolution On X-Transformed Points. Advances in Neural Information Processing Systems, 31, 820–830. https://proceedings.neurips.cc/paper_files/paper/2018/file/f5f8590cd58a54e94377e6ae2eded4d9-Paper.pdf
Chicago
Li, Y., R. Bu, M. Sun, W. Wu, X. Di, and B. Chen. 2018. “PointCNN: Convolution On X-Transformed Points”. Advances in Neural Information Processing Systems 31: 820–30. https://proceedings.neurips.cc/paper_files/paper/2018/file/f5f8590cd58a54e94377e6ae2eded4d9-Paper.pdf.
Harvard
Li, Y. et al. (2018) “PointCNN: Convolution On X-Transformed Points”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 820–830. Available at: https://proceedings.neurips.cc/paper_files/paper/2018/file/f5f8590cd58a54e94377e6ae2eded4d9-Paper.pdf.
Vancouver
1. Li Y, Bu R, Sun M, Wu W, Di X, Chen B (2018) PointCNN: Convolution On X-Transformed Points. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 820–830

BibTeX

@inproceedings{NEURIPS2018_f5f8590c,
 author = {Li, Yangyan and Bu, Rui and Sun, Mingchao and Wu, Wei and Di, Xinhan and Chen, Baoquan},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
 pages = {},
 publisher = {Curran Associates, Inc.},
 title = {PointCNN: Convolution On X-Transformed Points},
 url = {https://proceedings.neurips.cc/paper_files/paper/2018/file/f5f8590cd58a54e94377e6ae2eded4d9-Paper.pdf},
 volume = {31},
 year = {2018}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors