Multi-view 3D Object Detection Network for Autonomous Driving

Xiaozhi ChenHuimin MaJi WanBo LiTian Xia

article2017CVPR3,242 citations

Proposes MV3D, a sensory fusion framework that combines multi-view LiDAR representations with RGB images to generate precise 3D bounding boxes, outperforming prior autonomous driving detection methods on the KITTI benchmark by up to 30% average precision.

Listen

The paper addresses the challenge of accurate 3D object detection for autonomous driving, where vehicles must localize and classify road objects in three dimensions to ensure safe navigation. Existing LIDAR-only methods deliver strong 3D positioning but limited semantic detail, while image-only methods perform better in 2D yet struggle with precise depth and orientation. The work therefore set out to create a single network that fuses LIDAR point clouds with RGB images to produce oriented 3D bounding boxes at higher accuracy than either modality alone.

The authors developed MV3D, a two-stage architecture. A proposal network first generates 3D candidate boxes from a compact bird’s-eye-view encoding of the point cloud. A region-based fusion network then projects those proposals onto bird’s-eye, front-view, and image feature maps, combines the region features through a deep hierarchical fusion scheme, and jointly classifies objects while regressing their full 3D extent and orientation. The model was trained and tested end-to-end on the KITTI benchmark using both a LIDAR-only variant and a multimodal variant.

On the validation set the LIDAR-only version raised 3D localization average precision by roughly 25 percentage points and 3D detection average precision by roughly 30 points relative to prior LIDAR methods. Adding the RGB stream produced further gains, reaching 89 percent moderate 3D detection AP at an IoU threshold of 0.5. With only 300 proposals the network attained 99 percent recall at IoU 0.25 and 91 percent at IoU 0.5, far above competing 3D proposal generators. On the official test set the same approach improved 2D detection average precision by 10 points on the hard subset among all LIDAR-based entries while remaining competitive with leading image-only detectors.

These accuracy improvements translate directly to reduced collision risk and more reliable path planning, because vehicles can now obtain both precise 3D location and orientation from a single forward pass. The deep fusion design proved superior to conventional early or late fusion, confirming that intermediate-layer interactions across views add measurable value. The bird’s-eye view alone already outperformed the other single views, yet the full three-view combination delivered the highest scores, indicating complementary information across modalities.

The main limitations are the focus on the car category only, the 0.36-second inference time on a Titan X GPU, and reliance on the specific KITTI sensor configuration and annotation protocol. Results have not yet been demonstrated on additional datasets or under adverse weather that degrades LIDAR returns. Further work should therefore include real-time optimization, evaluation on broader object classes and datasets, and closed-loop testing on instrumented vehicles before deployment decisions can be made.

Baidu Tsinghua University autonomous driving multiple views object detection Multi-View 3D Object Detection MV3D LIDAR

arXiv: 1611.07759

Paper: Fast R-CNN, Ross B. Girshick (2015). Fast R-CNN establishes the region-of-interest pooling and multi-task loss framework that the source network adapts for multi-view feature fusion.
Paper: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, Shaoqing Ren et al. (2015). Faster R-CNN introduces the region proposal network architecture that forms the basis for the source paper's 3D candidate box generator.

Paper: VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection, Yin Zhou et al. (2017). VoxelNet directly succeeds this multi-view fusion framework by proposing an end-to-end architecture that learns 3D shape features directly from raw point clouds without hand-crafted representations.
Paper: PointPillars: Fast Encoders for Object Detection From Point Clouds, Alex H. Lang et al. (2018). PointPillars extends the efficient encoding concepts from this multi-view approach to vertical point cloud columns, enabling faster 3D object detection using standard 2D convolutions.

Cover for Multi-view 3D Object Detection Network for Autonomous Driving

Abstract

This paper aims at high-accuracy 3D object detection in autonomous driving scenario. We propose Multi-View 3D networks (MV3D), a sensory-fusion framework that takes both LIDAR point cloud and RGB images as input and predicts oriented 3D bounding boxes. We encode the sparse 3D point cloud with a compact multi-view representation. The network is composed of two subnetworks: one for 3D object proposal generation and another for multi-view feature fusion. The proposal network generates 3D candidate boxes efficiently from the bird's eye view representation of 3D point cloud. We design a deep fusion scheme to combine region-wise features from multiple views and enable interactions between intermediate layers of different paths. Experiments on the challenging KITTI benchmark show that our approach outperforms the state-of-the-art by around 25% and 30% AP on the tasks of 3D localization and 3D detection. In addition, for 2D detection, our approach obtains 10.3% higher AP than the state-of-the-art on the hard data among the LIDAR-based methods.

Topics

autonomous driving multiple views object detection point cloud rgb images viewpoint detection algorithm network architecture lidar systems

Keywords

Multi-View 3D Object Detection MV3D LIDAR RGB Image Bird’s Eye View Region-Based Fusion Network 3D Proposal Network Oriented 3D Box Regression KITTI Benchmark Deep Fusion

1. Introduction
2. Related Work
3. MV3D Network
3.1. 3D Point Cloud Representation
3.2. 3D Proposal Network
3.3. Region-based Fusion Network
3.4. Implementation
4. Experiments
5. Conclusion

Knowls

Knowl 1 — Multi-View 3D Object Detection Framework
model/method
The Multi-View 3D detection network (MV3D) is a sensory-fusion framework that takes raw LIDAR point clouds and RGB camera images to predict oriented 3D bounding boxes in autonomous driving scenarios. The framework consists of two sequential stages:
1. 3D Object Proposal Generation: Sparse 3D point cloud data is encoded into a Bird's Eye View (BV) representation. A 2D fully convolutional subnetwork generates oriented 3D bounding box proposals directly from the BV feature maps, exploiting the scale-invariance and non-overlapping spatial distribution of objects viewed from above.
2. Region-based Multi-View Feature Fusion: Each candidate 3D proposal is projected onto feature maps extracted from three distinct views: Bird's Eye View (BV), Front View (FV, a cylindrical range projection of the point cloud), and the RGB camera image plane. Region of Interest (ROI) pooling produces equal-length feature vectors for each view. A hierarchical deep fusion subnetwork integrates intermediate features across views to jointly predict the object classification category and perform oriented 3D bounding box regression.
Knowl 2 — Point Cloud Multi-View Feature Encoding
model/method
To preserve geometric detail without resorting to computationally intensive 3D voxel convolutions, 3D point clouds are projected into two complementary 2D multi-channel representations:
1. Bird's Eye View (BV) Representation: The 3D point cloud is projected onto the ground plane and discretized into a 2D grid with a resolution of $0.1\,\text{m}$ . The grid is encoded with $(M + 2)$ channels:
- Height Slices ( $M$ channels): The point cloud is divided vertically into $M$ equal height slices. For each grid cell, the height feature in a slice is the maximum $z$ -coordinate among all points within that cell and slice.
- Intensity Map ( $1$ channel): The reflectance intensity value of the point possessing the maximum height in each cell.
- Density Map ( $1$ channel): The point density normalized as $\min\left(1.0, \frac{\log(N + 1)}{\log(64)}\right)$ , where $N$ is the total count of points located in the cell.
1. Front View (FV) Representation: The point cloud is projected onto a cylindrical surface to generate a dense 2D front view map. A 3D point $p = (x, y, z)$ in the sensor coordinate frame maps to coordinates $p_{\text{fv}} = (r, c)$ via: $c = \lfloor \text{atan2}(y, x) / \Delta\theta \rfloor$ $r = \lfloor \text{atan2}(z, \sqrt{x^2 + y^2}) / \Delta\phi \rfloor$ where $\Delta\theta$ and $\Delta\phi$ denote the horizontal and vertical angular resolutions of the laser beams. The FV map encodes three channels: height ( $z$ ), radial distance ( $\sqrt{x^2 + y^2 + z^2}$ ), and reflectance intensity.
Knowl 3 — Bird's Eye View 3D Proposal Generation Network
model/method
The 3D Proposal Network generates oriented 3D candidate bounding boxes by applying 2D convolutions over the Bird's Eye View (BV) feature representation of the point cloud.

3D Prior Anchors: Each 3D prior box is defined by $(x, y, z, l, w, h)$ , representing center coordinates and bounding box dimensions in meters. Anchor dimensions $(l, w)$ are determined by clustering training ground truths (for cars: $(3.9, 1.6)\,\text{m}$ and $(1.0, 0.6)\,\text{m}$ ), with a fixed height $h = 1.56\,\text{m}$ . Anchors are evaluated at two orientations ( $0^\circ$ and $90^\circ$ ), yielding $N = 4$ prior boxes per grid location. Vertical centers $z$ are set based on sensor height and average object height.

Feature Resolution and Empty Anchor Filtering: The front-end backbone proceeds through three pooling operations ( $8\times$ downsampling) followed by a $2\times$ bilinear upsampling deconvolution layer, yielding a $4\times$ downsampled feature map relative to the BV input. Anchors containing zero LIDAR points are pre-filtered using an integral image computed over the point occupancy map.

Regression Targets and Loss: The network regresses offset vectors $t = (\Delta x, \Delta y, \Delta z, \Delta l, \Delta w, \Delta h)$ , where $(\Delta x, \Delta y, \Delta z)$ are center offsets normalized by anchor dimensions, and $\Delta s = \log(s_{\text{GT}} / s_{\text{anchor}})$ for $s \in \{l, w, h\}$ . The multi-task loss is: $\mathcal{L}_{\text{proposal}} = \mathcal{L}_{\text{cls}}(\text{objectness}) + \mathcal{L}_{\text{reg}}(\text{Smooth } L_1)$ Positive anchors are assigned where the 2D BV Intersection-over-Union (IoU) with ground truth is $\ge 0.7$ ; negative anchors have IoU $< 0.5$ . Non-Maximum Suppression (NMS) with an IoU threshold of 0.7 on BV boxes selects 2000 proposals for training and 300 proposals for inference.
Knowl 4 — Hierarchical Deep Fusion for Multi-View Features
model/method
Given a 3D candidate proposal $p_{\text{3D}}$ , Multi-View ROI pooling extracts fixed-length feature representations $f_v$ across three views: Bird's Eye View (BV), Front View (FV), and camera image plane (RGB): $f_v = R(x_v, \text{ROI}_v), \quad \text{ROI}_v = T_{\text{3D}\to v}(p_{\text{3D}}), \quad v \in \{\text{BV}, \text{FV}, \text{RGB}\}$ where $T_{\text{3D}\to v}$ projects 3D coordinates into the 2D coordinate frame of view $v$ , $x_v$ is the convolutional feature map of view $v$ , and $R(\cdot, \cdot)$ denotes ROI pooling.

To allow intermediate feature representations of all views to interact recursively across $L$ layers, Deep Fusion combines features hierarchically: $f_0 = f_{\text{BV}} \oplus f_{\text{FV}} \oplus f_{\text{RGB}}$ $f_l = H_l^{\text{BV}}(f_{l-1}) \oplus H_l^{\text{FV}}(f_{l-1}) \oplus H_l^{\text{RGB}}(f_{l-1}), \quad \forall l = 1, \dots, L$ where $H_l^v$ denotes the $l$ -th layer transformation function for view $v$ , and $\oplus$ is the element-wise mean join operation. This contrasts with early fusion, which merges views only at the input stage ( $f_L = H_L(\dots H_1(f_{\text{BV}} \oplus f_{\text{FV}} \oplus f_{\text{RGB}}))$ ), and late fusion, which processes each view independently until the final layer ( $f_L = H_L^{\text{BV}}(f_{\text{BV}}) \oplus H_L^{\text{FV}}(f_{\text{FV}}) \oplus H_L^{\text{RGB}}(f_{\text{RGB}})$ ).
Knowl 5 — 8-Corner Oriented 3D Bounding Box Regression
model/method
In the Region-based Fusion Network of MV3D, oriented 3D bounding boxes are predicted via an 8-corner offset parameterization instead of directly regressing center offsets, box dimensions, and yaw angles.

Parameterization: For each 3D candidate proposal, the regression target is a 24-dimensional vector specifying the offsets of all 8 bounding box corners: $t = (\Delta x_0, \dots, \Delta x_7, \Delta y_0, \dots, \Delta y_7, \Delta z_0, \dots, \Delta z_7)$ Each corner offset is normalized by the 3D diagonal length of the candidate proposal box $d_{\text{proposal}}$ : $\Delta x_i = \frac{x_i^{\text{GT}} - x_i^{\text{proposal}}}{d_{\text{proposal}}}, \quad \Delta y_i = \frac{y_i^{\text{GT}} - y_i^{\text{proposal}}}{d_{\text{proposal}}}, \quad \Delta z_i = \frac{z_i^{\text{GT}} - z_i^{\text{proposal}}}{d_{\text{proposal}}} \quad (i=0,\dots,7)$ Object orientations are computed directly from the predicted 3D corner coordinates.

Training and Inference:
- Loss: Multitask loss comprising softmax cross-entropy for object category classification and Smooth $L_1$ loss for the 24-dimensional corner offsets.
- Sampling: 3D proposals with Bird's Eye View (BV) IoU $\ge 0.5$ against ground-truth boxes are assigned as positive ROIs, and those with IoU $< 0.5$ are assigned as negative.
- Inference NMS: Predicted 3D boxes are projected to the BV plane, and Non-Maximum Suppression is applied using a strict IoU threshold of 0.05 to enforce that objects do not occupy overlapping physical space on the ground plane.
Knowl 6 — Fusion Network Regularization via Drop-Path and Auxiliary Losses
model/method
To regularize the Region-based Fusion Network and prevent co-adaptation among view streams, two complementary strategies are utilized during training:
1. Drop-Path Training: At each training iteration, the network selects either global drop-path or local drop-path with $50\%$ probability:
- Global Drop-Path: A single view is chosen uniformly at random from $\{\text{BV}, \text{FV}, \text{RGB}\}$ (each with probability $1/3$ ), and only the chosen view pathway is evaluated while the others are dropped.
- Local Drop-Path: At every join node, incoming paths from individual views are independently dropped with a probability of $50\%$ , with the constraint that at least one incoming path remains active.
1. Auxiliary Loss Regularization: Auxiliary classification and 3D bounding box regression branches are attached to each individual view stream ( $\{\text{BV}, \text{FV}, \text{RGB}\}$ ). The layers in these auxiliary paths share weights with the corresponding layers in the main network. Auxiliary branches are supervised using the same multi-task loss (classification cross-entropy plus Smooth $L_1$ corner regression) with equal weighting to the main loss. All auxiliary branches are removed during inference.
Knowl 7 — 3D Object Proposal Recall on KITTI Benchmark
empirical result
The 3D proposal generation network of MV3D, evaluated on the moderate difficulty regime of the KITTI validation set for the car category, achieves the following 3D bounding box recall metrics:
- At an Intersection-over-Union (IoU) threshold of $0.25$ , MV3D achieves $99.1\%$ 3D recall using 300 proposals.
- At an IoU threshold of $0.50$ , MV3D achieves $91.0\%$ 3D recall using 300 proposals.
In comparison, the stereo-based proposal method 3DOP achieves a maximum recall of $73.9\%$ at an IoU threshold of $0.50$ when using up to thousands of candidate proposals. Across all IoU thresholds and proposal counts, proposal generation from the Bird's Eye View representation outperforms monocular (Mono3D) and stereo (3DOP) proposal generation methods.

Knowl 8 — 3D Localization and 3D Object Detection Performance on KITTI

data/table

Performance comparison on the KITTI validation set (car category) for 3D localization Average Precision ( $\text{AP}_{\text{loc}}$ , evaluated on oriented bird's eye view boxes) and 3D object detection Average Precision ( $\text{AP}_{\text{3D}}$ , evaluated on oriented 3D boxes) across Easy, Moderate, and Hard regimes at an IoU threshold of 0.7:

Method	Data Modality	$\text{AP}_{\text{loc}}$ (IoU = 0.7) [%]			$\text{AP}_{\text{3D}}$ (IoU = 0.7) [%]
Method	Data Modality	Easy	Moderate	Hard	Easy	Moderate	Hard
Mono3D	Monocular	5.22	5.19	4.13	2.53	2.31	2.31
3DOP	Stereo	12.63	9.49	7.59	6.55	5.07	4.10
VeloFCN	LIDAR	40.14	32.08	30.47	15.20	13.66	15.98
MV3D (BV+FV)	LIDAR	86.18	77.32	76.33	71.19	56.60	55.30
MV3D (BV+FV+RGB)	LIDAR + Mono	86.55	78.10	76.67	71.29	62.68	56.56

At $\text{IoU} = 0.7$ , the LIDAR-only MV3D model (BV+FV) achieves $77.32\%\,\text{AP}_{\text{loc}}$ and $56.60\%\,\text{AP}_{\text{3D}}$ on Moderate data, outperforming the LIDAR-based baseline VeloFCN by $45.24\%$ on $\text{AP}_{\text{loc}}$ and $42.94\%$ on $\text{AP}_{\text{3D}}$ . Combining LIDAR and RGB images (BV+FV+RGB) further increases Moderate $\text{AP}_{\text{3D}}$ to $62.68\%$ .

Knowl 9 — Ablation Study on Multimodal Fusion Schemes and View Inputs

data/table

Ablation studies on the KITTI validation set (car category) comparing fusion network architectures (all using BV+FV+RGB) and combinations of input views (all using deep fusion):

Setting	$\text{AP}_{\text{3D}}$ (IoU=0.5) [%]			$\text{AP}_{\text{loc}}$ (IoU=0.5) [%]			$\text{AP}_{\text{2D}}$ (IoU=0.7) [%]
Setting	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
Fusion Architecture
Early Fusion	93.92	87.60	87.23	94.31	88.15	87.61	87.29	85.76	78.77
Late Fusion	93.53	87.70	86.88	93.84	88.12	87.20	87.47	85.36	78.66
Deep Fusion w/o aux. loss	94.21	88.29	87.21	94.57	88.75	88.02	88.64	85.74	79.06
Deep Fusion w/ aux. loss	96.02	89.05	88.38	96.34	89.39	88.67	95.01	87.59	79.90
View Combinations
FV	67.60	56.30	49.98	74.02	62.18	57.61	75.61	61.60	54.29
RGB	73.68	68.86	61.94	77.30	71.68	64.58	83.80	76.45	73.42
BV	92.30	85.50	78.94	92.90	86.98	86.14	85.00	76.21	74.80
FV + RGB	77.41	71.63	64.30	82.57	75.19	66.96	86.34	77.47	74.59
FV + BV	95.19	87.65	80.11	95.74	88.57	88.13	88.41	78.97	78.16
BV + RGB	96.09	88.70	80.52	96.45	89.19	80.69	89.61	87.76	79.76
BV + FV + RGB	96.02	89.05	88.38	96.34	89.39	88.67	95.01	87.59	79.90

The ablation results demonstrate:

Deep Fusion without auxiliary losses improves moderate $\text{AP}_{\text{3D}}$ over early and late fusion by $\sim 0.6\%$ . Adding auxiliary losses provides an additional $\sim 0.76\%$ improvement on moderate $\text{AP}_{\text{3D}}$ and an improvement of $+6.37\%$ on easy $\text{AP}_{\text{2D}}$ .
Among single views, Bird's Eye View (BV) is the most informative modality ( $85.50\%$ moderate $\text{AP}_{\text{3D}}$ ), while Front View (FV) is least effective on its own ( $56.30\%$ ). Every two-view combination outperforms the constituent individual views, and the complete three-view fusion achieves the best overall performance across all metrics.

Knowl 10 — 2D Object Detection Performance on KITTI Test Benchmark

data/table

2D object detection performance ( $\text{AP}_{\text{2D}}$ at $\text{IoU} = 0.7$ ) on the KITTI test set for the car category, comparing MV3D (where 2D detections are obtained by projecting predicted 3D bounding boxes onto the image plane) against existing 2D and 3D detection methods:

Method	Sensor Modality	Easy [%]	Moderate [%]	Hard [%]
Methods directly optimizing 2D bounding boxes
Faster R-CNN	Monocular	87.90	79.11	70.19
Mono3D	Monocular	90.27	87.86	78.09
3DOP	Stereo	90.09	88.34	78.79
MS-CNN	Monocular	90.46	88.83	74.76
SubCNN	Monocular	90.75	88.86	79.24
SDP+RPN	Monocular	89.90	89.42	78.54
Methods optimizing 3D bounding boxes
Vote3D	LIDAR	56.66	48.05	42.64
VeloFCN	LIDAR	70.68	53.45	46.90
Vote3Deep	LIDAR	76.95	68.39	63.22
3D FCN	LIDAR	85.54	75.83	68.30
MV3D (BV+FV)	LIDAR	89.80	79.76	78.61
MV3D (BV+FV+RGB)	LIDAR + Mono	90.37	88.90	79.81

Among LIDAR-based 3D detection models, MV3D (BV+FV) achieves $78.61\%\,\text{AP}_{\text{2D}}$ on the hard regime, outperforming 3D FCN by $10.31\%$ . When multimodal fusion (BV+FV+RGB) is used, MV3D achieves $79.81\%\,\text{AP}_{\text{2D}}$ on hard data, competitive with state-of-the-art 2D detectors that directly optimize 2D bounding boxes.

Coverage note — No substantial contributed material was omitted; all key methodology (multi-view point cloud encoding, 3D proposal generation, hierarchical deep fusion, 8-corner 3D box regression, and drop-path/auxiliary loss regularization) and empirical findings (3D proposal recall, 3D localization, 3D detection, 2D detection, and ablation studies) are fully represented.

References

1.Z. Cai, Q. Fan, R. Feris, and N. Vasconcelos. A unified multi-scale deep convolutional neural network for fast object detection. In ECCV, 2016.
2.J. Carreira and C. Sminchisescu. Cpmc: Automatic object segmentation using constrained parametric min-cuts. PAMI, 34(7):1312–1328, 2012.
3.X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun. Monocular 3d object detection for autonomous driving. In CVPR, 2016.
4.X. Chen, K. Kundu, Y. Zhu, A. Berneshawi, H. Ma, S. Fidler, and R. Urtasun. 3d object proposals for accurate object class detection. In NIPS, 2015.
5.X. Chen, K. Kundu, Y. Zhu, H. Ma, S. Fidler, and R. Urtasun. 3d object proposals using stereo imagery for accurate object class detection. In PAMI, 2017.
6.V. Dhiman, Q. H. Tran, J. J. Corso, and M. Chandraker. A continuous occlusion model for road scene understanding. In CVPR, pages 4331–4339, 2016.
7.M. Engelcke, D. Rao, D. Zeng Wang, C. Hay Tong, and I. Posner. Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks. arXiv:1609.06666, 2016.
8.M. Enzweiler and D. M. Gavrila. A multilevel mixture-of-experts framework for pedestrian classification. IEEE Transactions on Image Processing, 20(10):2967–2979, 2011.
9.A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012.
10.R. Girshick. Fast R-CNN. In ICCV, 2015.
11.A. Gonzalez, D. Vazquez, A. Lopez, and J. Amores. On-board object detection: Multicue, multimodal, and multiview random forest of local experts. In IEEE Transactions on Cybernetics, 2016.
12.V. Hegde and R. Zadeh. Fusionnet: 3d object classification using multiple data representations. CoRR, abs/1607.05695, 2016.
13.J. Hoffman, S. Gupta, and T. Darrell. Learning with side information through modality hallucination. In CVPR, 2016.
14.J. Hosang, R. Benenson, P. Dollár, and B. Schiele. What makes for effective detection proposals? PAMI, 2015.
15.G. Larsson, M. Maire, and G. Shakhnarovich. Fractalnet: Ultra-deep neural networks without residuals. arXiv:1605.07648, 2016.
16.B. Li. 3d fully convolutional network for vehicle detection in point cloud. IROS, 2017.
17.B. Li, T. Zhang, and T. Xia. Vehicle detection from 3d lidar using fully convolutional network. In Robotics: Science and Systems, 2016.
18.C. R. Qi, M. N. H. Su, A. Dai, M. Yan, and L. Guibas. Volumetric and multi-view cnns for object classification on 3d data. In CVPR, 2016.
19.S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
20.K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In arXiv:1409.1556, 2014.
21.S. Song and M. Chandraker. Joint sfm and detection cues for monocular 3d localization in road scenes. In Computer Vision and Pattern Recognition, pages 3734–3742, 2015.
22.S. Song and J. Xiao. Sliding shapes for 3d object detection in depth images. In ECCV. 2014.
23.S. Song and J. Xiao. Deep sliding shapes for amodal 3d object detection in rgb-d images. In CVPR, 2016.
24.H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. In ICCV, 2015.
25.K. Van de Sande, J. Uijlings, T. Gevers, and A. Smeulders. Segmentation as selective search for object recognition. In ICCV, 2011.
26.D. Z. Wang and I. Posner. Voting for voting in online point cloud object detection. In Proceedings of Robotics: Science and Systems, 2015.
27.J. Wang, Z. Wei, T. Zhang, and W. Zeng. Deeply-fused nets. arXiv:1605.07716, 2016.
28.Y. Xiang, W. Choi, Y. Lin, and S. Savarese. Data-driven 3d voxel patterns for object category recognition. In CVPR, 2015.
29.Y. Xiang, W. Choi, Y. Lin, and S. Savarese. Subcategory-aware convolutional neural networks for object proposals and detection. In arXiv:1604.04693. 2016.
30.F. Yang, W. Choi, and Y. Lin. Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers. In CVPR, 2016.
31.M. Z. Zia, M. Stark, B. Schiele, and K. Schindler. Detailed 3d representations for object recognition and modeling. PAMI, 2013.
32.M. Z. Zia, M. Stark, and K. Schindler. Are cars just 3d boxes? jointly estimating the 3d shape of multiple objects. In CVPR, pages 3678–3685, 2014.
33.L. Zitnick and P. Dollár. Edge boxes: Locating object proposals from edges. In ECCV. 2014.

Citation

MLA

Chen, X., et al. “Multi-View 3D Object Detection Network for Autonomous Driving”. arXiv, 2016, http://arxiv.org/abs/1611.07759v3.

APA

Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2016). Multi-View 3D Object Detection Network for Autonomous Driving. arXiv. http://arxiv.org/abs/1611.07759v3

Chicago

Chen, X., H. Ma, J. Wan, B. Li, and T. Xia. 2016. “Multi-View 3D Object Detection Network for Autonomous Driving”. arXiv. http://arxiv.org/abs/1611.07759v3.

Harvard

Chen, X. et al. (2016) “Multi-View 3D Object Detection Network for Autonomous Driving”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1611.07759v3.

Vancouver

1. Chen X, Ma H, Wan J, Li B, Xia T (2016) Multi-View 3D Object Detection Network for Autonomous Driving. arXiv

BibTeX

@article{chen2016multi,
  title = {Multi-View 3D Object Detection Network for Autonomous Driving},
  author = {Chen, Xiaozhi and Ma, Huimin and Wan, Ji and Li, Bo and Xia, Tian},
  year = {2016},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1611.07759v3},
  eprint = {1611.07759}
}

Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE

Multi-view 3D Object Detection Network for Autonomous Driving

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Multi-View 3D Object Detection Framework

Knowl 2 — Point Cloud Multi-View Feature Encoding

Knowl 3 — Bird's Eye View 3D Proposal Generation Network

Knowl 4 — Hierarchical Deep Fusion for Multi-View Features

Knowl 5 — 8-Corner Oriented 3D Bounding Box Regression

Knowl 6 — Fusion Network Regularization via Drop-Path and Auxiliary Losses

Knowl 7 — 3D Object Proposal Recall on KITTI Benchmark

Knowl 8 — 3D Localization and 3D Object Detection Performance on KITTI

Knowl 9 — Ablation Study on Multimodal Fusion Schemes and View Inputs

Knowl 10 — 2D Object Detection Performance on KITTI Test Benchmark

References

Citation

BibTeX

Access the Paper