Back

Center-based 3D Object Detection and Tracking

Tianwei YinXingyi ZhouPhilipp Krähenbühl

article2020CVPR2,389 citations

Proposes CenterPoint, an anchor-free framework that detects 3D objects as center points and simplifies tracking to closest-point matching, achieving state-of-the-art results on the nuScenes and Waymo Open Dataset benchmarks.

Listen

Reliable 3D perception is vital for autonomous driving systems to safely navigate complex environments. Traditional methods attempt to adapt two-dimensional bounding box techniques by aligning boxes to a fixed grid. However, three-dimensional objects in real driving situations can point in any direction, causing standard box-based detectors to struggle during critical maneuvers like turns or when handling objects of unusual shapes and sizes.

The article evaluates CenterPoint, a framework designed to represent, detect, and track 3D objects as orientation-free points rather than rigid bounding boxes. The objective is to demonstrate that a point-centered representation simplifies 3D recognition and tracking while delivering higher accuracy and lower computational overhead.

The authors tested this approach using large-scale automotive benchmark datasets, specifically the Waymo Open Dataset and the nuScenes dataset. The framework first identifies object center points and predicts attributes such as size, orientation, and velocity from an overhead view. An optional lightweight second stage refines these estimates by checking point features across the faces of the predicted 3D bounding box. Tracking is conducted simply by matching estimated object velocities across consecutive frames using closest-point distance matching, avoiding complex filtering systems.

The findings show that switching from box-based templates to a center-based representation improves detection accuracy by roughly 3 to 4 points across standard metrics. On the Waymo benchmark, CenterPoint surpassed previous single-model methods, achieving a 7.1% accuracy gain for vehicles and a 10.6% gain for pedestrians. For multi-object tracking, the framework outperformed prior state-of-the-art approaches by 8.8 points on nuScenes and by up to 50% on Waymo. Furthermore, the tracking module adds only 1 millisecond of computation, allowing the overall system to operate near real time at 11 to 16 frames per second.

These results demonstrate that point-based representations remove the heavy computational burdens and false positives associated with complex box alignment. The framework significantly enhances safety-critical performance, particularly for rotated objects, small objects like traffic cones, and elongated vehicles. The reduction in tracking complexity from dedicated Kalman filters to nearest-neighbor matching allows organizations to lower latency and compute costs without sacrificing tracking precision.

Engineering and perception teams developing autonomous vehicle systems should consider transitioning from anchor-based bounding boxes to point-based detection architectures. Adopting lightweight point-based tracking can immediately reduce compute pipelines. Where maximum precision is necessary and high-density sensor data is available, deploying the two-stage refinement module provides a favorable balance of higher accuracy with less than 10% additional computational cost.

The article notes limitations depending on sensor density. The two-stage refinement module provided significant gains on dense sensor data but showed no improvement on sparser 32-beam point clouds. Additionally, point-grid quantization can constrain refinement for very small objects like pedestrians on lower-resolution configurations. Nonetheless, given consistent top rankings across major public benchmarks, confidence in the framework's core detection and tracking performance remains high.

University of Texas at Austindetection algorithmobject detectionpoint cloudObjectDetectionObject Detection
arXiv: 2006.11275tianweiy/CenterPoint
  • Paper: Objects as Points, Xingyi Zhou et al. (2019). Objects as Points establishes the underlying keypoint-based object detection formulation that the source paper adapts and extends to 3D bounding boxes and tracking.
  • Paper: nuScenes: A Multimodal Dataset for Autonomous Driving, Holger Caesar et al. (2019). nuScenes provides the primary multimodal autonomous driving benchmark and evaluation metrics upon which the source paper measures and validates its state-of-the-art performance.
Cover for Center-based 3D Object Detection and Tracking

Abstract

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-the-art performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, CenterPoint outperforms all previous single model method by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at this https URL.

Topics

detection algorithmobject detectionpoint cloudtracking algorithmtracking method

Keywords

ObjectDetectionObject DetectionTrackingCenter-basedPointnuScenesBoxObjectsmAP

Table of Contents

  • 1 Introduction
  • 2 Related work
  • 3 Preliminaries
  • 4 CenterPoint
  • 4.1 Two-Stage CenterPoint
  • 4.2 Architecture
  • 5 Experiments
  • 5.1 Main Results
  • 5.2 Ablation studies
  • 6 Conclusion
  • References
  • A Tracking algorithm
  • B Implementation Details
  • C nuScenes Performance across classes
  • D nuScenes Detection Challenge

Knowls

  1. Knowl 1 — CenterPoint First-Stage 3D Object Detection Framework

    model/method

    CenterPoint represents 3D objects as point centers in bird's-eye view (BEV) and predicts bounding box properties directly from point features without requiring predefined anchor boxes.

    Given an irregular 3D point cloud P={(x,y,z,r)i}P = \{(x, y, z, r)_i\} with spatial coordinates (x,y,z)(x, y, z) and reflectance rr, a 3D backbone network (such as VoxelNet or PointPillars) quantizes the point cloud and converts it into a top-down BEV feature map M∈RW×H×FM \in \mathbb{R}^{W \times H \times F} with width WW, height HH, and channel dimension FF.

    A 2D convolutional detection head applied to MM produces dense predictions across several parallel branches:

    1. Center Heatmap Head: Outputs a category-specific heatmap Y^∈[0,1]W×H×K\hat{Y} \in [0, 1]^{W \times H \times K} for KK classes, where local maxima (peaks) indicate object centers.
    2. Sub-Voxel Location Offset Head: Regresses an offset o∈R2o \in \mathbb{R}^2 to compensate for quantization errors introduced by voxelization and stride operations.
    3. Elevation Head: Predicts height-above-ground hg∈Rh_g \in \mathbb{R} to localize the object in 3D along the vertical axis.
    4. 3D Dimension Head: Regresses bounding box dimensions as logarithmic scale s=(log⁡w,log⁡l,log⁡h)∈R3s = (\log w, \log l, \log h) \in \mathbb{R}^3.
    5. Yaw Orientation Head: Regresses continuous rotation values (sin⁡α,cos⁡α)∈R2(\sin \alpha, \cos \alpha) \in \mathbb{R}^2 from which yaw angle α\alpha is derived.
    6. Velocity Head: Predicts 2D ground-plane velocity v∈R2v \in \mathbb{R}^2 reflecting the displacement between consecutive time frames.

    All regression heads are supervised at ground-truth object center locations using an L1L_1 loss, while the center heatmap is supervised using a focal loss.

  2. Knowl 2 — Two-Stage Box Refinement via 5-Point BEV Feature Extraction

    model/method

    The second stage of CenterPoint refines proposal bounding boxes generated by the first-stage center detector by extracting point features from five surface centers of each predicted 3D bounding box.

    Because the top, bottom, and spatial center of a 3D bounding box project to the exact same point on a top-down bird's-eye view (BEV) map, CenterPoint extracts features for:

    • The predicted 3D bounding box center.
    • The 4 outward-facing vertical face centers (front, back, left, right).

    For each of the 5 points, features are sampled directly from the backbone's BEV feature map M∈RW×H×FM \in \mathbb{R}^{W \times H \times F} using bilinear interpolation. The five extracted feature vectors are concatenated into a single feature vector and passed through a shared two-layer Multi-Layer Perceptron (MLP) equipped with Batch Normalization, ReLU, and Dropout (dropout rate 0.3). The resulting representation feeds into two separate branches of three fully-connected layers:

    1. A confidence score branch predicting a class-agnostic confidence score I^∈[0,1]\hat{I} \in [0, 1].
    2. A box refinement branch predicting regression deltas (Δu,Δv,Δd,Δw,Δl,Δh,Δα)(\Delta u, \Delta v, \Delta d, \Delta w, \Delta l, \Delta h, \Delta \alpha) applied to the first-stage proposal.

    During training, 128 proposal boxes are sampled per frame with a 1:1 positive-to-negative ratio, where proposals having ≥0.55\ge 0.55 3D Intersection-over-Union (IoU) with ground truth are positive. At inference, the refinement runs on the top 500 first-stage proposals after Non-Maximum Suppression (NMS).

  3. Knowl 3 — Velocity-Guided Center-Based 3D Tracking Algorithm

    algorithm

    CenterPoint performs multi-object 3D tracking across time by combining object center detections with predicted 2D velocity vectors via greedy closest-distance matching.

    Input: Tracked objects from frame t - 1: T^{(t-1)} = {(p_j, v_j, c_j, q_j, id_j, a_j)}_{j=1}^M
           where p is center position, v is 2D velocity, c is category label, q is bounding box attributes,
           id is track ID, and a is inactive age (a = 0 for active tracks).
           Detections in frame t in descending confidence: \hat{D}^{(t)} = {(\hat{p}_i, \hat{v}_i, \hat{c}_i, \hat{q}_i)}_{i=1}^N.
           Matching distance threshold \tau_c for class c; Maximum inactive age threshold A (set to 3).
    Output: Tracked objects for frame t: T^{(t)}.
    Initialize T^{(t)} <- empty set, matched set S <- empty set
    Compute cost matrix F of size N x M:
      F_{ij} = ||\hat{p}_i^{(t)} - \hat{v}_i - p_j^{(t-1)}||_2
    for i = 1 to N do
        j <- arg min_{k \notin S} F_{ik}
        if F_{ij} <= \tau_{\hat{c}_i} then
            a_i^{(t)} <- 0
            T^{(t)} <- T^{(t)} \cup {(\hat{D}_i^{(t)}, id_j^{(t-1)}, a_i^{(t)})}
            S <- S \cup {j}
        else
            a_i^{(t)} <- 0
            T^{(t)} <- T^{(t)} \cup {(\hat{D}_i^{(t)}, newID, a_i^{(t)})}
        end if
    end for
    for j = 1 to M do
        if j \notin S then
            if T.a_j^{(t-1)} < A then
                T.a_j^{(t)} <- T.a_j^{(t-1)} + 1
                T.p_j^{(t)} <- T.p_j^{(t-1)} + T.v_j^{(t-1)}
                T^{(t)} <- T^{(t)} \cup {T_j^{(t-1)}}
            end if
        end if
    end for
    return T^{(t)}

    The matching associates current detections with past tracks by subtracting the predicted velocity v^i\hat{v}_i to project the center back to frame t−1t-1. Unmatched tracks persist up to A=3A=3 frames before deletion, with center positions updated by linear motion extrapolation.

  4. Knowl 4 — Dense Gaussian Supervision for BEV Center Heatmap Estimation

    equation

    Because bird's-eye view (BEV) projections preserve physical Euclidean distances, objects occupy a much smaller fraction of map pixels compared to perspective camera images, leading to sparse supervisory signals. CenterPoint enlarges the positive supervision region around each ground-truth object center in the target heatmap Y∈[0,1]W×H×KY \in [0, 1]^{W \times H \times K}.

    The target heatmap is rendered using a 2D Gaussian kernel at each annotated center location with standard deviation σ\sigma defined as:

    σ=max⁡(f(w,l),τ)\sigma = \max(f(w, l), \tau)

    where:

    • ww and ll are the width and length of the 3D object bounding box in meters,
    • f(w,l)f(w, l) is the bounding box radius function adapted from CornerNet,
    • τ=2\tau = 2 is the minimum allowable Gaussian radius in pixels.

    The heatmap head is supervised with the modified focal loss:

    Lheatmap=−1N∑x,y,c{(1−Y^xyc)αlog⁡(Y^xyc)if Yxyc=1(1−Yxyc)β(Y^xyc)αlog⁡(1−Y^xyc)otherwiseL_{\text{heatmap}} = -\frac{1}{N} \sum_{x,y,c} \begin{cases} (1 - \hat{Y}_{xyc})^\alpha \log(\hat{Y}_{xyc}) & \text{if } Y_{xyc} = 1 \\ (1 - Y_{xyc})^\beta (\hat{Y}_{xyc})^\alpha \log(1 - \hat{Y}_{xyc}) & \text{otherwise} \end{cases}

    where Y^xyc\hat{Y}_{xyc} is the predicted probability at location (x,y)(x, y) for class cc, NN is the number of objects, and α,β\alpha, \beta are focal loss hyperparameters.

  5. Knowl 5 — IoU-Guided Confidence Target and Geometric Score Ensembling

    equation

    In the two-stage CenterPoint model, the second-stage classification branch is supervised to predict an IoU-guided quality score rather than a hard binary classification label.

    For the tt-th 3D proposal bounding box having an Intersection-over-Union IoUt\text{IoU}_t with its matched ground-truth 3D box, the training target It∈[0,1]I_t \in [0, 1] is defined by:

    It=min⁡(1,max⁡(0,2×IoUt−0.5))I_t = \min(1, \max(0, 2 \times \text{IoU}_t - 0.5))

    The confidence head output I^t\hat{I}_t is supervised using binary cross-entropy loss:

    Lscore=−Itlog⁡(I^t)−(1−It)log⁡(1−I^t)L_{\text{score}} = -I_t \log(\hat{I}_t) - (1 - I_t) \log(1 - \hat{I}_t)

    At inference time, the final object confidence score Q^t\hat{Q}_t is computed as the geometric mean of the first-stage center heatmap peak score Y^t\hat{Y}_t (where Y^t=max⁡1≤k≤KY^t,k\hat{Y}_t = \max_{1 \le k \le K} \hat{Y}_{t, k}) and the second-stage score I^t\hat{I}_t:

    Q^t=Y^t⋅I^t\hat{Q}_t = \sqrt{\hat{Y}_t \cdot \hat{I}_t}

    This ensembled score simultaneously reflects category presence and 3D bounding box localization accuracy.

  6. Knowl 6 — Center-Based vs. Anchor-Based 3D Detection across Orientations and Sizes

    empirical result

    Replacing axis-aligned 2D anchors on bird's-eye view feature maps with a center-based keypoint representation substantially improves 3D object detection accuracy, particularly for rotated objects and objects with non-standard dimensions.

    On the Waymo Open Dataset validation set (evaluated by LEVEL 2 mAPH), anchor-based heads suffer severe performance degradation as the relative yaw angle of the ground truth box increases, whereas center-based heads maintain high precision across all orientation bins:

    Vehicle mAPH Pedestrian mAPH
    Relative Yaw 0∘−15∘0^\circ-15^\circ 15∘−30∘15^\circ-30^\circ 30∘−45∘30^\circ-45^\circ 0∘−15∘0^\circ-15^\circ 15∘−30∘15^\circ-30^\circ 30∘−45∘30^\circ-45^\circ
    Object Proportion 81.4%81.4\% 10.5%10.5\% 8.1%8.1\% 71.4%71.4\% 15.8%15.8\% 12.8%12.8\%
    Anchor-based 67.1 47.7 45.4 55.9 32.0 26.5
    Center-based 67.8 46.4 51.6 64.0 42.1 35.7

    Similarly, when instances are split into size tertiles (small 33%, medium 33%, large 33%), center-based detection improves small and large vehicle mAPH (small: 59.0 vs. 58.5; large: 65.4 vs. 64.4) and dramatically improves pedestrian mAPH across all scales (small: 38.5 vs. 29.6; medium: 69.5 vs. 60.2; large: 69.0 vs. 60.1). Overall on Waymo validation, center-based representation outperforms anchor-based by +4.3 mAPH with VoxelNet and +4.5 mAPH with PointPillars.

  7. Knowl 7 — 3D Detection and Tracking Performance on Waymo Open Dataset Benchmark

    data/table

    On the Waymo Open Dataset test benchmark, single-model CenterPoint (CenterPoint-Voxel) sets state-of-the-art performance for both 3D detection and 3D tracking without multi-model ensembling.

    Detection results on Waymo test set (IoU threshold 0.7 for vehicles, 0.5 for pedestrians):

    Difficulty Method Vehicle mAP Vehicle mAPH Pedestrian mAP Pedestrian mAPH
    LEVEL 1 StarNet 61.5 61.0 67.8 59.9
    PointPillars 63.3 62.8 62.1 50.2
    PPBA 67.5 67.0 69.7 61.7
    RCD 72.0 71.6 - -
    CenterPoint (Ours) 80.2 79.7 78.3 72.1
    LEVEL 2 StarNet 54.9 54.5 61.1 54.0
    PointPillars 55.6 55.1 55.9 45.1
    PPBA 59.6 59.1 63.0 55.8
    RCD 65.1 64.7 - -
    CenterPoint (Ours) 72.2 71.8 72.2 66.4

    Tracking results on Waymo test set (evaluated with Multiple Object Tracking Accuracy, MOTA, and Precision, MOTP):

    Difficulty Method Vehicle MOTA (% ↑\uparrow) Pedestrian MOTA (% ↑\uparrow) Vehicle MOTP (cm ↓\downarrow) Pedestrian MOTP (cm ↓\downarrow)
    LEVEL 1 AB3D Baseline 42.5 38.9 18.6 34.0
    CenterPoint (Ours) 62.6 58.3 16.3 31.1
    LEVEL 2 AB3D Baseline 40.1 37.7 18.6 34.0
    CenterPoint (Ours) 59.4 56.6 16.4 31.2

    CenterPoint improves LEVEL 2 mAPH over previous methods by 7.1% for vehicles and 10.6% for pedestrians, while improving tracking MOTA by up to 50% relative gain over Kalman filter baselines.

  8. Knowl 8 — 3D Detection and Tracking Performance on nuScenes Benchmark

    data/table

    On the nuScenes test set, CenterPoint achieves state-of-the-art results in both 3D detection and 3D multi-object tracking.

    nuScenes 3D detection test benchmark results (mean Average Precision mAP, nuScenes Detection Score NDS, and Planning KL Divergence metric PKL):

    Method mAP (% ↑\uparrow) NDS (% ↑\uparrow) PKL (↓\downarrow)
    WYSIWYG 35.0 41.9 1.14
    PointPillars 40.1 55.0 1.00
    CVCNet 55.3 64.4 0.92
    PointPainting 46.4 58.1 0.89
    PMPNet 45.4 53.1 0.81
    SSN 46.3 56.9 0.77
    CBGS 52.8 63.3 0.77
    CenterPoint (Ours) 58.0 65.5 0.69

    nuScenes 3D tracking test benchmark results (Average Multi-Object Tracking Accuracy AMOTA, False Positives FP, False Negatives FN, and ID Switches IDS):

    Method AMOTA (% ↑\uparrow) FP (↓\downarrow) FN (↓\downarrow) IDS (↓\downarrow)
    AB3D 15.1 15,088 75,730 9,027
    Chiu et al. 55.0 17,533 33,216 950
    CenterPoint (Ours) 63.8 18,612 22,928 760

    Ablation of the tracker on nuScenes validation demonstrates that given identical CenterPoint-Voxel detections, point velocity-based closest distance matching achieves 63.7 AMOTA in 1ms runtime, compared to 60.0 AMOTA in 73ms for a Mahalanobis distance Kalman filter tracker.

  9. Knowl 9 — Ablation of Second-Stage Point Feature Aggregation vs. 3D Voxel Features

    empirical result

    Evaluating different point sampling strategies and feature representations for the second-stage refinement on the Waymo validation set reveals that sampling sparse BEV features at the box and surface centers matches or exceeds the accuracy of complex 3D voxel feature extraction while being computationally faster.

    Comparison of sampling points from the 2D BEV feature map:

    Encoder Method Vehicle mAPH Pedestrian mAPH TproposalT_{\text{proposal}} TrefineT_{\text{refine}}
    VoxelNet First Stage Only 66.5 62.7 71ms -
    + Box Center (1 pt) 68.0 64.9 71ms 5ms
    + Surface Centers (5 pts) 68.3 65.3 71ms 6ms
    Dense Sampling (6×66 \times 6 pts) 68.2 65.4 71ms 8ms
    PointPillars First Stage Only 66.5 57.4 56ms -
    + Box Center (1 pt) 67.3 57.4 56ms 6ms
    + Surface Centers (5 pts) 67.5 57.9 56ms 7ms
    Dense Sampling (6×66 \times 6 pts) 67.3 57.9 56ms 8ms

    Comparison of BEV feature map interpolation against 3D voxel feature aggregations:

    • BEV Feature (5 points): 68.3 Vehicle mAPH, 65.3 Pedestrian mAPH, 77ms total runtime.
    • Voxel-Set Abstraction (VSA): 68.3 Vehicle mAPH, 65.2 Pedestrian mAPH, 98ms total runtime.
    • Radial Basis Function (RBF) Interpolation: 68.4 Vehicle mAPH, 65.7 Pedestrian mAPH, 89ms total runtime.

    Sampling 5 points from the BEV map achieves parity with 3D voxel aggregations while avoiding expensive raw voxel/point neighbor queries.

  10. Knowl 10 — CenterPoint Multi-Modal and Test-Time Ensembling Pipeline for NuScenes Challenge

    empirical result

    For the NeurIPS 2020 nuScenes 3D Detection Challenge, the authors extended CenterPoint using multi-modal point decoration and test-time augmentation, improving the baseline from 57.1 mAP and 65.4 NDS to 68.2 mAP and 71.7 NDS on the nuScenes validation split.

    The progression of improvements:

    1. PointPainting Fusion: Decorating each LiDAR point with image instance segmentation scores predicted by a Cascade R-CNN trained on nuImages (+5.6 mAP, +2.6 NDS).
    2. Double Flip Testing: Test-time horizontal and vertical coordinate flipping (+2.2 mAP, +1.4 NDS).
    3. Yaw Rotation Augmentation: Rotating point clouds at test time by [0∘,±6.25∘,±12.5∘,±25∘][0^\circ, \pm 6.25^\circ, \pm 12.5^\circ, \pm 25^\circ] (+1.3 mAP, +0.9 NDS).
    4. Model Ensemble: Ensembling five models trained with different voxel grid sizes ranging from 0.05m×0.05m0.05\text{m} \times 0.05\text{m} to 0.15m×0.15m0.15\text{m} \times 0.15\text{m} (+1.5 mAP, +1.1 NDS).
    5. Empty Box Filtering: Removing 3D box detections containing zero LiDAR points (+0.5 mAP, +0.3 NDS).

Coverage note — None was omitted; all key architectural components, equations, algorithms, empirical findings, and competition extension details are fully covered.

References

  1. 1.Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. RSS, 2019.
  2. 2.Philipp Bergmann, Tim Meinhardt, and Laura Leal-Taixe. Tracking without bells and whistles. ICCV, 2019.
  3. 3.Keni Bernardin, Alexander Elbs, and Rainer Stiefelhagen. Multiple object tracking performance metrics and evaluation in a smart room environment. Citeseer.
  4. 4.Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. Simple online and realtime tracking. ICIP, 2016.
  5. 5.Alex Bewley, Pei Sun, Thomas Mensink, Dragomir Anguelov, and Cristian Sminchisescu. Range conditioned dilated convolutions for scale invariant 3d object detection. arXiv preprint arXiv:2005.09927, 2020.
  6. 6.Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. CVPR, 2020.
  7. 7.Qi Chen, Lin Sun, Ernest Cheung, Kui Jia, and Alan Yuille. Every view counts: Cross-view consistency in 3d object detection with hybrid-cylindrical-spherical voxelization. NeurIPS, 2020.
  8. 8.Qi Chen, Lin Sun, Zhixin Wang, Kui Jia, and Alan Yuille. Object as hotspots: An anchor-free 3d object detection approach via firing of hotspots. ECCV, 2020.
  9. 9.Yilun Chen, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Fast point r-cnn. ICCV, 2019.
  10. 10.Hsu-kuang Chiu, Antonio Prioletti, Jie Li, and Jeannette Bohg. Probabilistic 3d multi-object tracking for autonomous driving. arXiv:2001.05673, 2020.
  11. 11.Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. ICCV, 2017.
  12. 12.Martin Engelcke, Dushyant Rao, Dominic Zeng Wang, Chi Hay Tong, and Ingmar Posner. Vote3deep: Fast object detection in 3d point clouds using efficient convolutional neural networks. ICRA, 2017.
  13. 13.Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. IJCV, 2010.
  14. 14.Runzhou Ge, Zhuangzhuang Ding, Yihan Hu, Yu Wang, Sijia Chen, Li Huang, and Yuan Li. Afdet: Anchor free one stage 3d object detection. arXiv preprint arXiv:2006.12671, 2020.
  15. 15.Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. CVPR, 2012.
  16. 16.Ross Girshick. Fast r-cnn. ICCV, 2015.
  17. 17.Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. CVPR, 2014.
  18. 18.Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. CVPR, 2018.
  19. 19.Sylvain Gugger. The 1cycle policy. https://sgugger.github.io/the-1cycle-policy.html, 2018.
  20. 20.Chenhang He, Hui Zeng, Jianqiang Huang, Xian-Sheng Hua, and Lei Zhang. Structure aware single-stage 3d object detection from point cloud. CVPR, 2020.
  21. 21.Kaiming He, Georgia Gkioxari, Piotr Doll'ar, and Ross Girshick. Mask r-cnn. ICCV, 2017.
  22. 22.Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. JMLR, 2012.
  23. 23.Peiyun Hu, Jason Ziglar, David Held, and Deva Ramanan. What you see is what you get: Exploiting visibility for 3d object detection. CVPR, 2020.
  24. 24.Rui Huang, Wanyue Zhang, Abhijit Kundu, Caroline Pantofaru, David A Ross, Thomas Funkhouser, and Alireza Fathi. An lstm approach to temporal 3d object detection in lidar point clouds. ECCV, 2020.
  25. 25.Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML, 2015.
  26. 26.Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, and Yuning Jiang. Acquisition of localization confidence for accurate object detection. ECCV, 2018.
  27. 27.H. Karunasekera, H. Wang, and H. Zhang. Multiple object tracking with attention to appearance, structure, motion and size. IEEE Access, 2019.
  28. 28.Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. CVPR, 2019.
  29. 29.Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. ECCV, 2018.
  30. 30.Buyu Li, Wanli Ouyang, Lu Sheng, Xingyu Zeng, and Xiaogang Wang. Gs3d: An efficient 3d object detection framework for autonomous driving. CVPR, 2019.
  31. 31.Ming Liang, Bin Yang, Yun Chen, Rui Hu, and Raquel Urtasun. Multi-task multi-sensor fusion for 3d object detection. CVPR, 2019.
  32. 32.Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. ICCV, 2017.
  33. 33.Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. ECCV, 2016.
  34. 34.Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. ICLR, 2019.
  35. 35.Mahyar Najibi, Guangda Lai, Abhijit Kundu, Zhichao Lu, Vivek Rathod, Thomas Funkhouser, Caroline Pantofaru, David Ross, Larry S Davis, and Alireza Fathi. Dops: Learning to detect 3d objects and predict their 3d shapes. CVPR, 2020.
  36. 36.Jiquan Ngiam, Benjamin Caine, Wei Han, Brandon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xi Yi, Ouais Alsharif, Patrick Nguyen, et al. Starnet: Targeted computation for object detection in point clouds. arXiv preprint arXiv:1908.11069, 2019.
  37. 37.Jonah Philion, Amlan Kar, and Sanja Fidler. Learning to evaluate perception models using planner-centric metrics. CVPR, 2020.
  38. 38.Charles R. Qi, Or Litany, Kaiming He, and Leonidas Guibas. Deep hough voting for 3d object detection in point clouds. ICCV, 2019.
  39. 39.Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb-d data. CVPR, 2018.
  40. 40.Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. CVPR, 2017.
  41. 41.Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS, 2017.
  42. 42.Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. CVPR, 2017.
  43. 43.Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NIPS, 2015.
  44. 44.Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. Pv-rcnn: Pointvoxel feature set abstraction for 3d object detection. CVPR, 2020.
  45. 45.Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointrcnn: 3d object proposal generation and detection from point cloud. CVPR, 2019.
  46. 46.Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. TPAMI, 2020.
  47. 47.Martin Simony, Stefan Milzy, Karl Amendey, and Horst-Michael Gross. Complex-yolo: An euler-region-proposal for real-time 3d object detection on point clouds. ECCV, 2018.
  48. 48.Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: An open dataset benchmark. CVPR, 2020.
  49. 49.Sourabh Vora, Alex H Lang, Bassam Helou, and Oscar Beijbom. Pointpainting: Sequential fusion for 3d object detection. CVPR, 2020.
  50. 50.Dequan Wang, Coline Devin, Qi-Zhi Cai, Philipp Krahenb ¨ uhl, ¨ and Trevor Darrell. Monocular plan view networks for autonomous driving. IROS, 2019.
  51. 51.Dominic Zeng Wang and Ingmar Posner. Voting for voting in online point cloud object detection. RSS, 2015.
  52. 52.Yue Wang, Alireza Fathi, Abhijit Kundu, David Ross, Caroline Pantofaru, Tom Funkhouser, and Justin Solomon. Pillarbased object detection for autonomous driving. ECCV, 2020.
  53. 53.Xinshuo Weng and Kris Kitani. A Baseline for 3D MultiObject Tracking. IROS, 2020.
  54. 54.Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with a deep association metric. ICIP, 2017.
  55. 55.Kelvin Wong, Shenlong Wang, Mengye Ren, Ming Liang, and Raquel Urtasun. Identifying unknown instances for autonomous driving. CORL, 2019.
  56. 56.Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embedded convolutional detection. Sensors, 2018.
  57. 57.Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real-time 3d object detection from point clouds. CVPR, 2018.
  58. 58.Xue Yang, Qingqing Liu, Junchi Yan, Ang Li, Zhiqiang Zhang, and Gang Yu. R3det: Refined single-stage detector with feature refinement for rotating object. arXiv:1908.05612, 2019.
  59. 59.Xue Yang, Jirui Yang, Junchi Yan, Yue Zhang, Tengfei Zhang, Zhi Guo, Xian Sun, and Kun Fu. Scrdet: Towards more robust detection for small, cluttered and rotated objects. ICCV, 2019.
  60. 60.Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia. 3dssd: Point-based 3d single stage object detector. CVPR, 2020.
  61. 61.Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Std: Sparse-to-dense 3d object detector for point cloud. ICCV, 2019.
  62. 62.Junbo Yin, Jianbing Shen, Chenye Guan, Dingfu Zhou, and Ruigang Yang. Lidar-based online 3d video object detection with graph-based message passing and spatiotemporal transformer attention. CVPR, 2020.
  63. 63.Xingyi Zhou, Vladlen Koltun, and Philipp Krahenb ¨ uhl. Tracking objects as points. ECCV, 2020.
  64. 64.Xingyi Zhou, Dequan Wang, and Philipp Krahenb ¨ uhl. Objects ¨ as points. arXiv:1904.07850, 2019.
  65. 65.Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Tom Ouyang, James Guo, Jiquan Ngiam, and Vijay Vasudevan. End-to-end multi-view fusion for 3d object detection in lidar point clouds. CORL, 2019.
  66. 66.Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. CVPR, 2018.
  67. 67.Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection. arXiv:1908.09492, 2019.
  68. 68.Xinge Zhu, Yuexin Ma, Tai Wang, Yan Xu, Jianping Shi, and Dahua Lin. Ssn: Shape signature networks for multi-class object detection from point clouds. ECCV, 2020.

Citation

MLA
Yin, T., et al. “Center-based 3D Object Detection and Tracking”. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 11779–88, https://doi.org/10.1109/CVPR46437.2021.01161.
APA
Yin, T., Zhou, X., & Krahenbuhl, P. (2021). Center-based 3D Object Detection and Tracking. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11779–11788. https://doi.org/10.1109/CVPR46437.2021.01161
Chicago
Yin, T., X. Zhou, and P. Krahenbuhl. 2021. “Center-based 3D Object Detection and Tracking”. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11779–88. https://doi.org/10.1109/CVPR46437.2021.01161.
Harvard
Yin, T., Zhou, X. and Krahenbuhl, P. (2021) “Center-based 3D Object Detection and Tracking”, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 11779–11788. Available at: https://doi.org/10.1109/CVPR46437.2021.01161.
Vancouver
1. Yin T, Zhou X, Krahenbuhl P (2021) Center-based 3D Object Detection and Tracking. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 11779–11788

BibTeX

@inproceedings{Yin_2021, title={Center-based 3D Object Detection and Tracking}, url={http://dx.doi.org/10.1109/CVPR46437.2021.01161}, DOI={10.1109/cvpr46437.2021.01161}, booktitle={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, publisher={IEEE}, author={Yin, Tianwei and Zhou, Xingyi and Krahenbuhl, Philipp}, year={2021}, month=June, pages={11779–11788} }
Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF