Back

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Qingyong HuBo YangLinhai XieStefano RosaYulan GuoZhihua WangNiki TrigoniAndrew Markham

article2019CVPR2,111 citations

Presents RandLA-Net, a lightweight neural network that couples random point sampling with local feature aggregation to perform semantic segmentation on million-point 3D point clouds up to 200 times faster than existing methods.

Listen

Real-time intelligent applications such as autonomous driving and augmented reality rely on 3D point cloud sensors to accurately perceive and segment complex physical surroundings. However, analyzing large-scale point clouds containing millions of points across broad outdoor spaces remains a major operational bottleneck. Existing deep learning methods depend heavily on computationally expensive sampling routines, graph construction, or multi-step preprocessing such as spatial partitioning and voxelization. These heavy requirements prevent current networks from scaling efficiently or operating at speeds required for real-time deployment.

The article sets out to design and validate an efficient, lightweight neural architecture that directly predicts per-point semantic labels from massive 3D point clouds in a single pass without any preprocessing. To achieve this, the authors introduce a framework called RandLA-Net, which combines simple random downsampling with a dedicated local feature aggregation module to maintain geometric accuracy.

The approach was evaluated through computational benchmark experiments and performance comparisons on three large public datasets: the outdoor benchmarks Semantic3D and SemanticKITTI, and the indoor dataset S3DIS. The core architecture uses random sampling to achieve fast, constant-time downsampling across successive neural layers. To prevent the loss of critical geometric details caused by random point removal, the architecture incorporates local spatial encoding to preserve relative 3D positions, an attentive pooling mechanism to weight important local features, and dilated residual blocks that expand each point’s receptive field across multi-hop neighborhoods. Baseline models and ablated configurations were tested on standard computing hardware to assess runtime speed, memory consumption, parameter counts, and segmentation accuracy.

The findings show substantial gains in processing speed, computational efficiency, and accuracy. First, random point sampling took only 0.004 seconds to downsample one million points, whereas traditional heuristic methods required 10 to 200 seconds and alternative learning-based sampling took up to 1,200 seconds or demanded prohibitive memory. Second, the full RandLA-Net architecture processed large-scale scans at roughly 22 frames per second, running up to 200 times faster than competing graph-based pipelines while consuming only 1.24 million parameters. Third, the system outperformed state-of-the-art methods on large-scale outdoor benchmarks, achieving 77.4% mean Intersection-over-Union on Semantic3D and 53.9% on SemanticKITTI. Ablation studies confirmed that stacking local spatial encoding units with attentive pooling was essential, as removing the dilated residual structure decreased segmentation accuracy by over 8 percentage points.

These results demonstrate that organizations can deploy high-accuracy 3D semantic perception directly onto edge computing hardware without incurring prohibitive processing delays or expensive preprocessing workflows. By replacing complex point-selection heuristics with random sampling supported by local feature aggregation, developers can significantly lower system memory footprints and hardware costs while ensuring high-frame-rate performance in safety-critical automated driving workflows.

Organizations developing autonomous navigation and spatial computing systems should consider transitioning from computationally heavy point selection and voxelization pipelines to lightweight random sampling architectures. Future development should focus on extending this architecture toward end-to-end 3D instance segmentation and real-time processing of continuous dynamic point cloud streams.

The findings carry high confidence based on extensive evaluations across established public benchmarks. However, decision-makers should note specific operational limitations: the system exhibited reduced accuracy on small, highly imbalanced object classes (such as bicycles and motorcyclists) and occasionally misclassified visually similar vehicles when color data was unavailable.

National University of Defense TechnologySun Yat-sen UniversityUniversity of Oxfordgenetic selectionlocal featurepoint cloudPointPoint CloudsPoints
arXiv: 1911.11236QingyongHu/RandLA-Net
  • Paper: KPConv: Flexible and Deformable Convolution for Point Clouds, Hugues Thomas et al. (2019). KPConv extends the lightweight philosophy of the source by introducing flexible point convolutions that further improve semantic segmentation accuracy on massive 3D scenes.
  • Paper: Point Transformer, Nico Engel et al. (2020). Point Transformer builds upon efficient point-based representations like the source by applying attention mechanisms to capture complex geometric structures in large point clouds.
Cover for RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Abstract

We study the problem of efficient semantic segmentation for large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Extensive experiments show that our RandLA-Net can process 1 million points in a single pass with up to 200X faster than existing approaches. Moreover, our RandLA-Net clearly surpasses state-of-the-art approaches for semantic segmentation on two large-scale benchmarks Semantic3D and SemanticKITTI.

Topics

genetic selectionlocal featurepoint cloudsemanticssyntacticsarchitecture designs

Keywords

PointPoint CloudsPointsSamplingRandLA-NetLarge-Scale PointPoint CloudLocalLarge-Scale Point CloudsSemantic

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 RandLA-Net
  • 3.1 Overview
  • 3.2 The quest for efficient sampling
  • 3.3 Local Feature Aggregation
  • 3.4 Implementation
  • 4 Experiments
  • 4.1 Efficiency of Random Sampling
  • 4.2 Efficiency of RandLA-Net
  • 4.3 Semantic Segmentation on Benchmarks
  • 4.4 Ablation Study
  • 5 Conclusion
  • References
  • A Details for the Evaluation of Sampling.
  • B Details of the Network Architecture
  • C Additional Ablation Studies on LocSE
  • D Additional Ablation Studies on Dilated Residual Block
  • E Visualization of Attention Scores
  • F Additional Results on Semantic3D
  • G Additional Results on SemanticKITTI
  • H Additional Results on S3DIS
  • I Video Illustration

Knowls

  1. Knowl 1 — RandLA-Net Architecture for Large-Scale Point Cloud Semantic Segmentation

    model/method

    RandLA-Net is an efficient, lightweight point-based neural network designed to directly perform per-point semantic segmentation on large-scale 3D point clouds (containing millions of points over hundreds of meters) in a single pass without voxelization, block partitioning, or graph construction.

    The network follows an encoder-decoder architecture with skip connections:

    1. Input: A point cloud of size N×dinN \times d_{\text{in}}, where NN is the number of points and dind_{\text{in}} is the input feature dimension (e.g., 3D coordinates plus RGB color, or 3D coordinates alone). An initial shared Multi-Layer Perceptron (MLP) extracts per-point features (N,8)(N, 8).
    2. Encoding Layers: Four successive encoding stages progressively downsample the points and expand feature channels. Each layer applies a Local Feature Aggregation (LFA) module followed by Random Sampling (RS) with a 4-fold decimation ratio (N→N/4→N/16→N/64→N/256N \to N/4 \to N/16 \to N/64 \to N/256), while channel dimensions increase as 8→32→128→256→5128 \to 32 \to 128 \to 256 \to 512.
    3. Decoding Layers: Four decoding stages restore spatial resolution. For each query point, nearest-neighbor interpolation using 1-nearest neighbor via KNN transfers features to the higher resolution. The interpolated features are concatenated via skip connections with intermediate features from the corresponding encoder stage, followed by shared MLPs that reduce feature dimensions ((N/64,256)→(N/16,128)→(N/4,32)→(N,8)(N/64, 256) \to (N/16, 128) \to (N/4, 32) \to (N, 8)).
    4. Prediction Head: Per-point class logits (N,nclass)(N, n_{\text{class}}) are computed via three shared fully-connected layers ((N,64)→(N,32)→(N,nclass))((N, 64) \to (N, 32) \to (N, n_{\text{class}})) and a dropout layer with a dropout ratio of 0.50.5.

    During training, point clouds are sub-sampled to approximately 10510^5 points per batch, optimized using Adam with an initial learning rate of 0.010.01 decaying by 5%5\% per epoch, and neighborhood size K=16K = 16. At inference, the entire raw point cloud is processed directly.

  2. Knowl 2 — Local Spatial Encoding Unit in RandLA-Net

    model/method

    The Local Spatial Encoding (LocSE) unit explicitly embeds the relative spatial coordinates of local point neighborhoods into point feature representations, enabling the network to learn local geometric structures.

    Given a center point pi∈R3p_i \in \mathbb{R}^3 with feature fi∈Rdf_i \in \mathbb{R}^d and its KK nearest neighbors {pi1,…,pik,…,piK}⊂R3\{p_i^1, \dots, p_i^k, \dots, p_i^K\} \subset \mathbb{R}^3 found via Euclidean distance KNN:

    1. Relative Point Position Encoding: The spatial relationships are encoded into a vector rikr_i^k using a shared MLP: rik=MLP(pi⊕pik⊕(pi−pik)⊕∥pi−pik∥)r_i^k = \text{MLP}\Big(p_i \oplus p_i^k \oplus (p_i - p_i^k) \oplus \|p_i - p_i^k\|\Big) where ⊕\oplus denotes vector concatenation and ∥pi−pik∥\|p_i - p_i^k\| is the Euclidean distance between center point pip_i and neighbor pikp_i^k.

    2. Point Feature Augmentation: The spatial encoding vector rikr_i^k is concatenated with the existing neighbor feature vector fik∈Rdf_i^k \in \mathbb{R}^d: f^ik=rik⊕fik\hat{f}_i^k = r_i^k \oplus f_i^k

    The output of LocSE for point pip_i is the augmented local feature set F^i={f^i1,…,f^ik,…,f^iK}\hat{F}_i = \{\hat{f}_i^1, \dots, \hat{f}_i^k, \dots, \hat{f}_i^K\}, which explicitly couples 3D geometric positioning with learned point semantics.

  3. Knowl 3 — Attentive Pooling Unit in RandLA-Net

    model/method

    The Attentive Pooling unit aggregates the set of local neighbor features F^i={f^i1,…,f^iK}\hat{F}_i = \{\hat{f}_i^1, \dots, \hat{f}_i^K\} for a center point pip_i using a learnable soft attention mechanism rather than fixed heuristic pooling operations (such as max or average pooling).

    1. Attention Score Computation: A shared function g(⋅,W)g(\cdot, W) consisting of a shared MLP followed by a softmax activation computes an attention score scalar siks_i^k for each neighbor feature f^ik\hat{f}_i^k: sik=g(f^ik,W)=softmax(MLP(f^ik))s_i^k = g(\hat{f}_i^k, W) = \text{softmax}\Big(\text{MLP}(\hat{f}_i^k)\Big) where WW denotes the learnable parameters of the shared MLP.

    2. Weighted Summation: The aggregated representation f~i\tilde{f}_i for center point pip_i is computed as the attention-weighted sum of neighbor features: f~i=∑k=1K(f^ik⋅sik)\tilde{f}_i = \sum_{k=1}^K \Big(\hat{f}_i^k \cdot s_i^k\Big)

    This dynamic soft mask retains important features within the local context and geometric configuration while suppressing redundant or noisy information.

  4. Knowl 4 — Dilated Residual Block for Neighborhood Receptive Field Expansion

    model/method

    To counter the potential loss of spatial detail caused by random point decimation, RandLA-Net chains multiple Local Spatial Encoding (LocSE) and Attentive Pooling units within a residual block with a skip connection.

    In each dilated residual block, two LocSE and Attentive Pooling stages are stacked sequentially:

    • The first LocSE-Attentive Pooling stage aggregates features from the KK-nearest spatial neighbors (1-hop neighborhood) of each point.
    • The second stage aggregates features across the KK-nearest neighbors of those neighbors, effectively expanding the receptive field of each point to up to K2K^2 points (2-hop neighborhood).

    The output of the chained units is combined with the input feature via a skip connection containing a shared MLP projection and LeakyReLU activation. This design dilates the receptive field cheaply without incurring the computational overhead of large neighborhood searches or dense graph construction.

  5. Knowl 5 — Computational and Memory Complexity of Point Sampling Strategies

    theoretical result

    An analysis of point downsampling methods for selecting KK points from a large-scale point cloud PP with NN points (N≈106N \approx 10^6) establishes the following computational and memory characteristics:

    1. Farthest Point Sampling (FPS): Time complexity is O(N2)\mathcal{O}(N^2). Sampling 10%10\% of 10610^6 points takes approximately 200200 seconds on a GPU, making it unsuitable for large-scale point clouds.
    2. Inverse Density Importance Sampling (IDIS): Ranks points by local inverse density with computational complexity O(N)\mathcal{O}(N). Empirically takes ≈10\approx 10 seconds for 10610^6 points and is sensitive to outliers.
    3. Random Sampling (RS): Uniformly selects KK point indices with computational complexity O(1)\mathcal{O}(1), independent of NN. It requires 00 extra GPU memory and executes in approximately 0.0040.004 seconds for 10610^6 points.
    4. Generator-based Sampling (GS): Uses a learned network to synthesize a subset but requires FPS at inference to match points, taking ≈1200\approx 1200 seconds for 10%10\% of 10610^6 points.
    5. Continuous Relaxation Sampling (CRS): Relaxes sampling into continuous weights via Gumbel-Softmax; performing one-pass matrix multiplication over 10610^6 points requires an estimated memory footprint exceeding 300300 GB.
    6. Policy Gradient Sampling (PGS): Formulates sampling as a Markov decision process with an action space of (106105)\binom{10^6}{10^5}, leading to extreme gradient variance and training non-convergence.

  6. Knowl 6 — Inference Efficiency and Parameter Benchmark on SemanticKITTI

    data/table

    The computational runtime, network parameter count, and maximum inference point capacity were evaluated on Sequence 08 (4071 scans) of the SemanticKITTI dataset using an AMD 3700X CPU and NVIDIA RTX 2080Ti GPU. For standardized timing, 81,920 points per scan were fed into each network.

    Method Total time (seconds) Parameters (millions) Maximum inference points (millions)
    PointNet (Vanilla) 192 0.8 0.49
    PointNet++ (SSG) 9831 0.97 0.98
    PointCNN 8142 11.0 0.05
    SPG 43584 0.25 -
    KPConv 717 14.9 0.54
    RandLA-Net (Ours) 185 1.24 1.03

    RandLA-Net processed all 4071 scans in 185 seconds (approximately 22 frames per second), operating up to 200×200\times faster than SPG and significantly faster than PointNet++ and PointCNN (which are bottlenecked by FPS). RandLA-Net can process up to 1.03×1061.03 \times 10^6 points in a single forward pass on an 11GB GPU.

  7. Knowl 7 — Semantic3D Benchmark Evaluation

    data/table

    Semantic segmentation performance of RandLA-Net on the online Semantic3D (reduced-8) test benchmark, evaluated using Mean Intersection-over-Union (mIoU, %) and Overall Accuracy (OA, %):

    Method mIoU (%) OA (%) man-made natural high veg. low veg. buildings hard scape scanning art. cars
    SnapNet 59.1 88.6 82.0 77.3 79.7 22.9 91.1 18.4 37.3 64.4
    SEGCloud 61.3 88.1 83.9 66.0 86.0 40.5 91.1 30.9 27.5 64.3
    RF_MSSF 62.7 90.3 87.6 80.3 81.8 36.4 92.2 24.1 42.6 56.6
    MSDeepVoxNet 65.3 88.4 83.0 67.2 83.8 36.7 92.4 31.3 50.0 78.2
    ShellNet 69.3 93.2 96.3 90.4 83.9 41.0 94.2 34.7 43.9 70.2
    GACNet 70.8 91.9 86.4 77.7 88.5 60.6 94.2 37.3 43.5 77.8
    SPG 73.2 94.0 97.4 92.6 87.9 44.0 83.2 31.0 63.5 76.2
    KPConv 74.6 92.9 90.9 82.2 84.2 47.9 94.9 40.0 77.3 79.7
    RandLA-Net (Ours) 77.4 94.8 95.6 91.4 86.6 51.5 95.7 51.5 69.8 76.8

    RandLA-Net achieved 77.4%77.4\% mIoU and 94.8%94.8\% OA on Semantic3D (reduced-8), outperforming all prior published methods in both overall metrics.

  8. Knowl 8 — SemanticKITTI Semantic Segmentation Benchmark Evaluation

    data/table

    Quantitative semantic segmentation results evaluated on the single-scan online test track of SemanticKITTI across 19 categories:

    Method mIoU road s-walk park o-grnd build car truck b-cle m-cle o-veh veg trunk terr person b-clist m-clist fence pole sign
    PointNet 14.6 61.6 35.7 15.8 1.4 41.4 46.3 0.1 1.3 0.3 0.8 31.0 4.6 17.6 0.2 0.2 0.0 12.9 2.4 3.7
    SPG 17.4 45.0 28.5 0.6 0.6 64.3 49.3 0.1 0.2 0.2 0.8 48.9 27.2 24.6 0.3 2.7 0.1 20.8 15.9 0.8
    SPLATNet 18.4 64.6 39.1 0.4 0.0 58.3 58.2 0.0 0.0 0.0 0.0 71.1 9.9 19.3 0.0 0.0 0.0 23.1 5.6 0.0
    PointNet++ 20.1 72.0 41.8 18.7 5.6 62.3 53.7 0.9 1.9 0.2 0.2 46.5 13.8 30.0 0.9 1.0 0.0 16.9 6.0 8.9
    TangentConv 40.9 83.9 63.9 33.4 15.4 83.4 90.8 15.2 2.7 16.5 12.1 79.5 49.3 58.1 23.0 28.4 8.1 49.0 35.8 28.5
    SqueezeSeg 29.5 85.4 54.3 26.9 4.5 57.4 68.8 3.3 16.0 4.1 3.6 60.0 24.3 53.7 12.9 13.1 0.9 29.0 17.5 24.5
    SqueezeSegV2 39.7 88.6 67.6 45.8 17.7 73.7 81.8 13.4 18.5 17.9 14.0 71.8 35.8 60.2 20.1 25.1 3.9 41.1 20.2 36.3
    DarkNet21Seg 47.4 91.4 74.0 57.0 26.4 81.9 85.4 18.6 26.2 26.5 15.6 77.6 48.4 63.6 31.8 33.6 4.0 52.3 36.0 50.0
    DarkNet53Seg 49.9 91.8 74.6 64.8 27.9 84.1 86.4 25.5 24.5 32.7 22.6 78.3 50.1 64.0 36.2 33.6 4.7 55.0 38.9 52.2
    RangeNet53++ 52.2 91.8 75.2 65.0 27.8 87.4 91.4 25.7 25.7 34.4 23.0 80.5 55.1 64.6 38.3 38.8 4.8 58.6 47.9 55.9
    RandLA-Net 53.9 90.7 73.7 60.3 20.4 86.9 94.2 40.1 26.0 25.8 38.9 81.4 61.3 66.8 49.2 48.2 7.2 56.3 49.2 47.7

    RandLA-Net achieved 53.9%53.9\% mIoU with 1.241.24M parameters, outperforming prior point-based models (PointNet 14.6%14.6\%, PointNet++ 20.1%20.1\%, TangentConv 40.9%40.9\%) and 2D projection-based models (RangeNet53++ 52.2%52.2\%, DarkNet53Seg 49.9%49.9\%) while using 40×40\times fewer parameters than RangeNet53++.

  9. Knowl 9 — S3DIS Indoor Semantic Segmentation Benchmark Evaluation

    data/table

    Semantic segmentation results on the Stanford 3D Indoor Spaces (S3DIS) dataset evaluated under standard 6-fold cross-validation across all 6 areas and 13 semantic categories:

    Method OA (%) mAcc (%) mIoU (%) ceil. floor wall beam col. wind. door table chair sofa book. board clut.
    PointNet 78.6 66.2 47.6 88.0 88.7 69.3 42.4 23.1 47.5 51.6 54.1 42.0 9.6 38.2 29.4 35.2
    RSNet - 66.5 56.5 92.5 92.8 78.6 32.8 34.4 51.6 68.1 59.7 60.1 16.4 50.2 44.9 52.0
    3P-RNN 86.9 - 56.3 92.9 93.8 73.1 42.5 25.9 47.6 59.2 60.4 66.7 24.8 57.0 36.7 51.6
    SPG 86.4 73.0 62.1 89.9 95.1 76.4 62.8 47.1 55.3 68.4 73.5 69.2 63.2 45.9 8.7 52.9
    PointCNN 88.1 75.6 65.4 94.8 97.3 75.8 63.3 51.7 58.4 57.2 71.6 69.1 39.1 61.2 52.2 58.6
    PointWeb 87.3 76.2 66.7 93.5 94.2 80.8 52.4 41.3 64.9 68.1 71.4 67.1 50.3 62.7 62.2 58.5
    ShellNet 87.1 - 66.8 90.2 93.6 79.9 60.4 44.1 64.9 52.9 71.6 84.7 53.8 64.6 48.6 59.4
    KPConv - 79.1 70.6 93.6 92.4 83.1 63.9 54.3 66.1 76.6 57.8 64.0 69.3 74.9 61.3 60.3
    RandLA-Net 88.0 82.0 70.0 93.1 96.1 80.6 62.4 48.0 64.4 69.4 69.4 76.4 60.0 64.2 65.9 60.1

    RandLA-Net achieved 70.0%70.0\% mIoU, 88.0%88.0\% Overall Accuracy, and 82.0%82.0\% mean class accuracy, processing entire rooms directly without needing to partition rooms into 1×11 \times 1 meter blocks.

  10. Knowl 10 — Ablation Study on Local Feature Aggregation and Pooling Mechanisms

    data/table

    Ablation studies evaluated the impact of individual architectural components of the Local Feature Aggregation module on SemanticKITTI (trained on sequences 00–07 and 09–10, validated on Sequence 08):

    Ablated Configuration mIoU (%)
    (1) Remove local spatial encoding (LocSE) 49.8
    (2) Replace attentive pooling with max-pooling 55.2
    (3) Replace attentive pooling with mean-pooling 53.4
    (4) Replace attentive pooling with sum-pooling 54.3
    (5) Simplify dilated residual block (single unit) 48.8
    (6) Full framework (RandLA-Net) 57.1

    Key findings:

    • Simplifying the dilated residual block to a single LocSE + Attentive Pooling unit reduced mIoU by 8.3%8.3\% (from 57.1%57.1\% to 48.8%48.8\%), demonstrating that expanding the receptive field to K2K^2 neighbors is critical when paired with random sampling.
    • Removing the LocSE unit caused a 7.3%7.3\% drop in mIoU (to 49.8%49.8\%), indicating that explicit relative coordinate embedding is essential for capturing local geometric context.
    • Replacing attentive pooling with max-, mean-, or sum-pooling degraded mIoU by 1.9%1.9\% to 3.7%3.7\%.

  11. Knowl 11 — Ablation on LocSE Spatial Position Embeddings and Residual Block Depth

    data/table

    Ablations on SemanticKITTI Sequence 08 tested different relative spatial encodings in the LocSE unit and varying numbers of chained aggregation units in the dilated residual block.

    Spatial Information Encodings in LocSE:

    LocSE Formulation mIoU (%)
    (1) (pi)(p_i) only 45.5
    (2) (pik)(p_i^k) only 47.7
    (3) (pi,pik)(p_i, p_i^k) 49.1
    (4) (pi,pik,∥pi−pik∥)(p_i, p_i^k, \|p_i - p_i^k\|) 50.5
    (5) (pi,pik,pi−pik)(p_i, p_i^k, p_i - p_i^k) 53.6
    (6) (pi,pik,pi−pik,∥pi−pik∥)(p_i, p_i^k, p_i - p_i^k, \|p_i - p_i^k\|) (Full unit) 54.3

    Relative position difference pi−pikp_i - p_i^k is the most critical component, increasing mIoU from 49.1%49.1\% to 53.6%53.6\%.

    Number of Aggregation Units in Residual Block:

    Chained Units per Block mIoU (%)
    (1) One aggregation unit 49.8
    (2) Two aggregation units (Standard Block) 54.3
    (3) Three aggregation units 51.1

    Two units perform best; three units cause performance to degrade from 54.3%54.3\% to 51.1%51.1\% due to parameter over-fitting.

  12. Knowl 12 — Limitations on Minority Classes and Geometric Ambiguity

    limitation

    RandLA-Net exhibits specific performance limitations in two main scenarios:

    1. Severe Class Imbalance: Segmentation accuracy drops significantly on extreme minority classes in outdoor point clouds, such as bicycle (26.0%26.0\% IoU), motorcycle (25.8%25.8\% IoU), bicyclist (48.2%48.2\% IoU), and motorcyclist (7.2%7.2\% IoU on SemanticKITTI). In these datasets, dominant classes like vegetation contain up to 7000 times more points than minority classes like motorcyclist.
    2. Geometric Ambiguity Without Color: In LiDAR-only datasets lacking RGB information (e.g., SemanticKITTI), objects with similar underlying 3D geometry and partial spatial scans (such as other-vehicle vs. car) are frequently confused.

Coverage note — None was omitted; all core architectural components, theoretical and empirical sampling comparisons, benchmark evaluations (Semantic3D, SemanticKITTI, S3DIS), ablation studies, and qualitative limitations are fully covered.

References

  1. 1.Abubakar Abid, Muhammad Fatih Balin, and James Zou. Concrete autoencoders for differentiable feature selection and reconstruction. In ICML, 2019.
  2. 2.Iro Armeni, Sasha Sax, Amir R Zamir, and Silvio Savarese. Joint 2D-3D-semantic data for indoor scene understanding. In CVPR, 2017.
  3. 3.Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Juergen Gall. SemanticKITTI: A dataset for semantic scene understanding of lidar sequences. In ICCV, 2019.
  4. 4.Alexandre Boulch, Bertrand Le Saux, and Nicolas Audebert. Unstructured point cloud semantic labeling using deep segmentation networks. In 3DOR, 2017.
  5. 5.Chao Chen, Guanbin Li, Ruijia Xu, Tianshui Chen, Meng Wang, and Liang Lin. ClusterNet: Deep hierarchical cluster network with rigorously rotation-invariant representation for point cloud analysis. In CVPR, 2019.
  6. 6.Lin-Zhuo Chen, Xuan-Yi Li, Deng-Ping Fan, Ming-Ming Cheng, Kai Wang, and Shao-Ping Lu. LSANet: Feature learning on point sets by local spatial attention. arXiv preprint arXiv:1905.05442, 2019.
  7. 7.Siheng Chen, Sufeng Niu, Tian Lan, and Baoan Liu. PCT: Large-scale 3D point cloud representations via graph inception networks with applications to autonomous driving. In ICIP, 2019.
  8. 8.Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3D object detection network for autonomous driving. In CVPR, 2017.
  9. 9.Yilun Chen, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Fast point R-CNN. In ICCV, 2019.
  10. 10.Christopher Choy, JunYoung Gwak, and Silvio Savarese. 4D spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR, 2019.
  11. 11.Chin Seng Chua and Ray Jarvis. Point signatures: A new representation for 3D object recognition. IJCV, 1997.
  12. 12.Oren Dovrat, Itai Lang, and Shai Avidan. Learning to sample. In CVPR, 2019.
  13. 13.Francis Engelmann, Theodora Kontogianni, and Bastian Leibe. Dilated point convolutions: On the receptive field of point convolutions. In BMVC, 2019.
  14. 14.Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3D semantic segmentation with submanifold sparse convolutional networks. In CVPR, 2018.
  15. 15.Fabian Groh, Patrick Wieschollek, and Hendrik P. A. Lensch. Flex-convolution (million-scale point-cloud learning beyond grid-worlds). In ACCV, 2018.
  16. 16.Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. arXiv preprint arXiv:1912.12033, 2019.
  17. 17.Timo Hackel, Nikolay Savinov, Lubor Ladicky, Jan D Wegner, Konrad Schindler, and Marc Pollefeys. Semantic3d. net: A new large-scale point cloud classification benchmark. ISPRS, 2017.
  18. 18.Timo Hackel, Jan D Wegner, and Konrad Schindler. Fast semantic segmentation of 3d point clouds with strongly varying density. ISPRS, 2016.
  19. 19.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
  20. 20.Binh-Son Hua, Minh-Khoi Tran, and Sai-Kit Yeung. Pointwise convolutional neural networks. In CVPR, 2018.
  21. 21.Qiangui Huang, Weiyue Wang, and Ulrich Neumann. Recurrent slice networks for 3D segmentation of point clouds. In CVPR, 2018.
  22. 22.Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, and Jiaya Jia. Hierarchical point-edge interaction network for point cloud semantic segmentation. In ICCV, 2019.
  23. 23.Artem Komarichev, Zichun Zhong, and Jing Hua. A-CNN: Annularly convolutional neural networks on point clouds. In CVPR, 2019.
  24. 24.Shiyi Lan, Ruichi Yu, Gang Yu, and Larry S Davis. Modeling local geometric structure of 3D point clouds using Geo-CNN. In CVPR, 2019.
  25. 25.Loic Landrieu, Hugo Raguet, Bruno Vallet, Clement Mallet,´ and Martin Weinmann. A structured regularization framework for spatially smoothing semantic labelings of 3d point clouds. ISPRS, 2017.
  26. 26.Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In CVPR, 2018.
  27. 27.Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. PointPillars: Fast encoders for object detection from point clouds. In CVPR, 2019.
  28. 28.Truc Le and Ye Duan. PointGrid: A deep network for 3D shape understanding. In CVPR, 2018.
  29. 29.Huan Lei, Naveed Akhtar, and Ajmal Mian. Octree guided cnn with spherical kernels for 3D point clouds. In CVPR, 2019.
  30. 30.Bo Li, Tianlei Zhang, and Tian Xia. Vehicle detection from 3D lidar using fully convolutional network. In RSS, 2016.
  31. 31.Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. Deepgcns: Can gcns go as deep as cnns? In ICCV, October 2019.
  32. 32.Jiaxin Li, Ben M Chen, and Gim Hee Lee. SO-Net: Self-organizing network for point cloud analysis. In CVPR, 2018.
  33. 33.Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. PointCNN: Convolution on X-transformed points. In NeurIPS, 2018.
  34. 34.Jinxian Liu, Bingbing Ni, Caiyuan Li, Jiancheng Yang, and Qi Tian. Dynamic points agglomeration for hierarchical point sets learning. In ICCV, 2019.
  35. 35.Xingyu Liu, Mengyuan Yan, and Jeannette Bohg. MeteorNet: Deep learning on dynamic 3D point cloud sequences. In ICCV, 2019.
  36. 36.Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In CVPR, 2019.
  37. 37.Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han. Point-voxel cnn for efficient 3d deep learning. In NeurIPS, 2019.
  38. 38.Jiageng Mao, Xiaogang Wang, and Hongsheng Li. Interpolated convolutional networks for 3D point cloud understanding. In ICCV, 2019.
  39. 39.Hsien-Yu Meng, Lin Gao, Yu-Kun Lai, and Dinesh Manocha. VV-net: Voxel vae net with group convolutions for point cloud segmentation. In ICCV, 2019.
  40. 40.Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. RangeNet++: Fast and accurate lidar semantic segmentation. In IROS, 2019.
  41. 41.Andriy Mnih and Karol Gregor. Neural variational inference and learning in belief networks. arXiv preprint arXiv:1402.0030, 2014.
  42. 42.Anshul Paigwar, Ozgur Erkent, Christian Wolf, and Christian Laugier. Attentional pointnet for 3d-object detection in point clouds. In CVPRW, 2019.
  43. 43.Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. In CVPR, 2017.
  44. 44.Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS, 2017.
  45. 45.Dario Rethage, Johanna Wald, Jurgen Sturm, Nassir Navab, and Federico Tombari. Fully-convolutional point networks for large-scale point clouds. In ECCV, 2018.
  46. 46.Xavier Roynard, Jean-Emmanuel Deschaud, and Franc¸ois Goulette. Classification of point cloud scenes with multi-scale voxel deep network. arXiv preprint arXiv:1804.03583, 2018.
  47. 47.Radu Bogdan Rusu, Nico Blodow, and Michael Beetz. Fast point feature histograms (fpfh) for 3D registration. In ICRA, 2009.
  48. 48.Yiru Shen, Chen Feng, Yaoqing Yang, and Dong Tian. Mining point cloud local structures by kernel correlation and graph pooling. In CVPR, 2018.
  49. 49.Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, and Jan Kautz. SPLATNet: sparse lattice networks for point cloud processing. In CVPR, 2018.
  50. 50.Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In NeurIPS, 2000.
  51. 51.Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou. Tangent convolutions for dense prediction in 3D. In CVPR, 2018.
  52. 52.Lyne Tchapmi, Christopher Choy, Iro Armeni, JunYoung Gwak, and Silvio Savarese. Segcloud: Semantic segmentation of 3D point clouds. In 3DV, 2017.
  53. 53.Hugues Thomas, Franc¸ois Goulette, Jean-Emmanuel Deschaud, and Beatriz Marcotegui. Semantic classification of 3D point clouds with multiscale spherical neighborhoods. In 3DV, 2018.
  54. 54.Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, Franc¸ois Goulette, and Leonidas J Guibas. KPConv: Flexible and deformable convolution for point clouds. In ICCV, 2019.
  55. 55.Chu Wang, Babak Samari, and Kaleem Siddiqi. Local spectral graph convolution for point set feature learning. In ECCV, 2018.
  56. 56.Lei Wang, Yuchun Huang, Yaolin Hou, Shenman Zhang, and Jie Shan. Graph attention convolution for point cloud semantic segmentation. In CVPR, 2019.
  57. 57.Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, and Justin M. Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG), 2019.
  58. 58.Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer. Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3D lidar point cloud. In ICRA, 2018.
  59. 59.Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer. Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud. In ICRA, 2019.
  60. 60.Wenxuan Wu, Zhongang Qi, and Li Fuxin. PointConv: Deep convolutional networks on 3D point clouds. In CVPR, 2018.
  61. 61.Saining Xie, Sainan Liu, Zeyu Chen, and Zhuowen Tu. Attentional shapecontextnet for point cloud recognition. In CVPR, 2018.
  62. 62.Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In ICML, 2015.
  63. 63.Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real-time 3D object detection from point clouds. In CVPR, 2018.
  64. 64.Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning object bounding boxes for 3D instance segmentation on point clouds. In NeurIPS, 2019.
  65. 65.Bo Yang, Sen Wang, Andrew Markham, and Niki Trigoni. Robust attentional aggregation of deep feature sets for multi-view 3D reconstruction. IJCV, 2019.
  66. 66.Jiancheng Yang, Qiang Zhang, Bingbing Ni, Linguo Li, Jinxian Liu, Mengdie Zhou, and Qi Tian. Modeling point clouds with self-attention and gumbel subset sampling. In CVPR, 2019.
  67. 67.Xiaoqing Ye, Jiamao Li, Hexiao Huang, Liang Du, and Xiaolin Zhang. 3D recurrent neural networks with context fusion for point cloud semantic segmentation. In ECCV, 2018.
  68. 68.Wenxiao Zhang and Chunxia Xiao. PCAN: 3D attention map learning using contextual information for point cloud based retrieval. In CVPR, 2019.
  69. 69.Zhiyuan Zhang, Binh-Son Hua, and Sai-Kit Yeung. Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics. In ICCV, 2019.
  70. 70.Hengshuang Zhao, Li Jiang, Chi-Wing Fu, and Jiaya Jia. Pointweb: Enhancing local neighborhood features for point cloud processing. In CVPR, 2019.

Citation

MLA
Hu, Q., et al. “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds”. arXiv, 2019, http://arxiv.org/abs/1911.11236v3.
APA
Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., & Markham, A. (2019). RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds. arXiv. http://arxiv.org/abs/1911.11236v3
Chicago
Hu, Q., B. Yang, L. Xie, et al. 2019. “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds”. arXiv. http://arxiv.org/abs/1911.11236v3.
Harvard
Hu, Q. et al. (2019) “RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1911.11236v3.
Vancouver
1. Hu Q, Yang B, Xie L, Rosa S, Guo Y, Wang Z, Trigoni N, Markham A (2019) RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds. arXiv

BibTeX

@article{hu2019randla,
  title = {RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds},
  author = {Hu, Qingyong and Yang, Bo and Xie, Linhai and Rosa, Stefano and Guo, Yulan and Wang, Zhihua and Trigoni, Niki and Markham, Andrew},
  year = {2019},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1911.11236v3},
  eprint = {1911.11236}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE