A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Haoqiang FanHao SuLeonidas Guibas

article2017CVPR2,736 citations

Introduces the Point Set Generation Network, a deep learning framework that reconstructs 3D object geometries directly as unordered point clouds from single 2D images while handling shape ambiguity by predicting multiple plausible outputs.

Listen

The article addresses the challenge of reconstructing complete 3D object shapes from single 2D images, a task that remains ill-posed due to missing depth information and inherent ambiguities in possible completions. Existing deep learning approaches rely on volumetric grids or image collections, which introduce quantization artifacts, limit resolution, and fail to respect natural geometric invariances under transformations.

The work set out to demonstrate that a neural network can directly generate unordered 3D point clouds as output, while also producing multiple plausible reconstructions to reflect ground-truth uncertainty.

The authors developed a conditional generative network with an encoder that processes the input image and a random vector, followed by predictor branches that combine fully connected layers for flexibility with deconvolution layers for smooth surfaces. They trained the model end-to-end using Chamfer distance or Earth Mover’s distance as loss functions on 220,000 synthetic models from ShapeNet, rendered into single-view images, and incorporated a min-of-N loss or conditional variational autoencoder to enable sampling of varied outputs.

The network consistently outperformed the prior state-of-the-art volumetric method 3D-R2N2 across CD, EMD, and intersection-over-union metrics on 13 object categories, often exceeding that baseline even when the baseline received five input views. It preserved thin structures such as chair legs and object details more reliably, successfully completed partial RGBD inputs by hallucinating missing geometry using learned shape priors, and generated multiple distinct yet plausible point clouds whose variation aligned with sources of ambiguity such as object thickness or unseen sides. Human modelers achieved comparable numerical scores on a validation subset, indicating the results approach manual reconstruction quality.

These outcomes indicate that point-cloud generation offers a more efficient and structurally faithful representation for single-image 3D recovery than volumetric alternatives, potentially lowering computational cost and improving downstream tasks that require surface geometry. The approach also provides a practical mechanism for capturing prediction uncertainty without explicit multi-view input.

Further development should focus on scaling to cluttered real-world scenes, integrating detection mechanisms for composite objects, and cascading local refinement steps to add high-frequency surface detail. Additional labeled real imagery and expanded category coverage would strengthen generalization before deployment in robotics or augmented-reality pipelines.

The primary limitations are reliance on synthetic training data with clean backgrounds, a fixed output of 1024 points, and occasional failures on unfamiliar shapes or heavily occluded objects; results on real photographs remain promising but unquantified at scale.

Stanford University Tsinghua University 3d reconstruction geometric transformations image generation Point Set Generation Network 3D object reconstruction point cloud generation

arXiv: 1612.00603 fanhqme/PointSetGeneration

Paper: 3D ShapeNets: A deep representation for volumetric shapes, Zhirong Wu et al. (2014). This seminal work introduced 3D deep shape representations and benchmarks using volumetric grids, providing the baseline paradigms and limitations that point set generation explicitly aims to overcome.
Paper: Multi-view Convolutional Neural Networks for 3D Shape Recognition, Hang Su et al. (2015). This paper establishes standard multi-view rendering and 3D shape classification benchmarks that form key methodological baselines for evaluating neural representations of 3D geometry.
Paper: Depth Map Prediction from a Single Image using a Multi-Scale Deep Network, David Eigen et al. (2014). This foundational work demonstrates multi-scale deep learning architectures for recovering 3D spatial depth maps directly from single RGB images.
Paper: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Alec Radford et al. (2016). This foundational architecture established deep convolutional generative models and deconvolutional branches that conditional generative shape networks adapt.
Paper: Conditional Image Generation with PixelCNN Decoders, Aäron van den Oord et al. (2016). This paper develops key methods for conditioning generative decoders on latent vectors and input features to generate diverse, plausible structured outputs.

Paper: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, Charles R. Qi et al. (2017). PointNet provides the foundational architecture for directly processing, classifying, and segmenting the unordered 3D point sets generated by point-based reconstruction pipelines.
Paper: PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, C. Qi et al. (2017). PointNet++ builds upon point set learning by introducing hierarchical local-neighborhood feature extraction for irregular 3D point clouds.
Paper: Occupancy Networks: Learning 3D Reconstruction in Function Space, Lars Mescheder et al. (2018). Occupancy Networks advance beyond discrete point sets to continuous implicit function representations for single-image 3D reconstruction.
Paper: DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation, Jeong Joon Park et al. (2019). DeepSDF generalizes learning-based 3D shape reconstruction by representing object geometry as continuous signed distance functions conditioned on latent codes.
Paper: PointCNN: Convolution On X-Transformed Points, Yangyan Li et al. (2018). PointCNN introduces generalized convolutions on unordered 3D point sets to improve hierarchical feature learning on raw geometric point data.
Paper: PCT: Point cloud transformer, Meng-Hao Guo et al. (2020). Point Cloud Transformer applies self-attention mechanisms directly to unordered 3D point clouds to model global and local spatial dependencies.
Paper: Deep Learning for 3D Point Clouds: A Survey, Yulan Guo et al. (2019). This comprehensive survey categorizes and contextualizes the broader ecosystem of deep learning techniques operating directly on raw 3D point clouds.
Paper: Zero-1-to-3: Zero-shot One Image to 3D Object, Ruoshi Liu et al. (2023). Zero-1-to-3 extends single-image 3D object reconstruction to zero-shot generalization across open-domain objects using diffusion priors.
Paper: LRM: Large Reconstruction Model for Single Image to 3D, Yicong Hong et al. (2024). LRM scales single-image 3D reconstruction into large-scale feed-forward transformer models capable of predicting full 3D radiance fields in seconds.

Cover for A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Abstract

Generation of 3D data by deep neural network has been attracting increasing attention in the research community. The majority of extant works resort to regular representations such as volumetric grids or collection of images; however, these representations obscure the natural invariance of 3D shapes under geometric transformations and also suffer from a number of other issues. In this paper we address the problem of 3D reconstruction from a single image, generating a straight-forward form of output -- point cloud coordinates. Along with this problem arises a unique and interesting issue, that the groundtruth shape for an input image may be ambiguous. Driven by this unorthodox output form and the inherent ambiguity in groundtruth, we design architecture, loss function and learning paradigm that are novel and effective. Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image. In experiments not only can our system outperform state-of-the-art methods on single image based 3d reconstruction benchmarks; but it also shows a strong performance for 3d shape completion and promising ability in making multiple plausible predictions.

Topics

3d reconstruction geometric transformations image generation neural networks point cloud reconstruction single images

Keywords

Point Set Generation Network 3D object reconstruction point cloud generation single view reconstruction Point Set Prediction shape completion ShapeNet dataset volumetric representation variational autoencoder

1 Introduction
2 Related Work
3 Problem and Notations
4 Approach
4.1 Overview
4.2 Point Set Prediction Network
4.3 Distance Metric between Point Sets
4.4 Generation of Multiple Plausible Shapes
5 Experiment
5.1 Training Data Generation by Synthesis
5.2 3D Shape Reconstruction from RGB Images
5.3 3D Shape Completion from RGBD Images
5.4 Predicting Multiple Plausible Shapes
5.5 Network Design Analysis
5.6 More results and application to real world data
5.7 Analysis of human ability for single view 3D reconstruction
5.8 Analysis of failure cases
5.9 Implementation details
6 Discussion
References

Knowls

Knowl 1 — Dual-Branch Point Set Generation Network Architecture
model/method
The Point Set Generation Network generates an unordered set of $N = 1024$ 3D point coordinates $S = \{(x_i, y_i, z_i)\}_{i=1}^N \in \mathbb{R}^{N \times 3}$ directly from a single 2D RGB or RGB-D image $I \in \mathbb{R}^{192 \times 256 \times C}$ (optionally perturbed by a random vector $r \sim \mathcal{N}(0, \mathbf{I})$ ).

The network comprises an encoder and a dual-branch predictor:
1. Encoder: A series of convolutional and ReLU layers with strided convolutions (16 feature channels at the highest resolution, doubling at each downsampling step) that projects the input image $I$ (and latent vector $r$ ) into a latent embedding.
2. Fully Connected (FC) Predictor Branch: A multi-layer fully connected network that predicts the 3D coordinates of $N_1 = 256$ points. Because each point is parameterized independently, this branch provides high flexibility to capture thin, detached, or intricate geometry (such as chair legs, plane tails, and firearm barrels).
3. Deconvolution (Deconv) Predictor Branch: A deconvolutional network that outputs a 3-channel feature map of dimensions $H \times W = 32 \times 24$ , yielding $H \times W = 768$ 3D points $(x, y, z)$ . By exploiting 2D spatial continuity and convolution weight sharing, this branch generates smooth continuous surfaces that wrap around the object's primary body.
4. Hourglass Extension: An extended architecture nests the encoder-decoder structure inside recurrent stacked hourglass modules with skip connections linking intermediate encoder feature maps to predictor layers, facilitating the fusion of global semantics with local structural details.
The final point cloud is the union of the $N_1$ points from the fully connected branch and the $H \times W$ points from the deconvolution branch ( $256 + 768 = 1024$ points).
Knowl 2 — Chamfer Distance for Point Set Prediction Loss
equation
The Chamfer Distance (CD) between two point sets $S_1, S_2 \subset \mathbb{R}^3$ is defined as:

$d_{CD}(S_1, S_2) = \sum_{x \in S_1} \min_{y \in S_2} \|x - y\|_2^2 + \sum_{y \in S_2} \min_{x \in S_1} \|x - y\|_2^2$

where $\|\cdot\|_2$ denotes the Euclidean norm in $\mathbb{R}^3$ .

Properties:
- $d_{CD}$ is non-negative and symmetric with respect to $S_1$ and $S_2$ , though it does not satisfy the triangle inequality.
- As a function of point coordinates in $S_1$ and $S_2$ , $d_{CD}$ is continuous and piecewise smooth, making it differentiable almost everywhere and suitable for backpropagation in neural networks.
- Nearest neighbor searches for each point $x \in S_1$ (or $y \in S_2$ ) are mutually independent, enabling trivial GPU parallelization and efficient calculation via spatial data structures (e.g., KD-trees).
Knowl 3 — Earth Mover's Distance for Point Set Prediction Loss
equation
The Earth Mover's Distance (EMD) between two point sets $S_1, S_2 \subset \mathbb{R}^3$ of equal cardinality $|S_1| = |S_2| = N$ is defined as the solution to the linear assignment problem:

$d_{EMD}(S_1, S_2) = \min_{\phi: S_1 \to S_2} \sum_{x \in S_1} \|x - \phi(x)\|_2$

where $\phi : S_1 \to S_2$ is a bijective mapping (permutation).

Properties:
- For almost all point set configurations (outside a zero-measure set), the optimal assignment bijection $\phi$ is unique and invariant to infinitesimal perturbations of point coordinates, rendering $d_{EMD}$ differentiable almost everywhere.
- Because exact assignment is computationally prohibitive for batch training on GPUs, an asynchronous $(1 + \epsilon)$ -approximation relaxation scheme with bounded runtime per instance is utilized, yielding gradient updates with assignment error typically around $1\%$ .
Knowl 4 — Min-of-N Loss for Modeling Ground Truth Ambiguity
model/method
To handle single-view 3D reconstruction ambiguity (where occluded/invisible regions admit multiple plausible geometric completions), the point set generator $G(I, r; \Theta)$ is parameterized by network weights $\Theta$ , conditioning image $I$ , and a latent perturbation vector $r \sim \mathcal{N}(0, \mathbf{I})$ .

Direct minimization of standard point set distances with a single random vector forces the network to ignore the latent code and predict an averaged mean shape. To preserve multi-modal diversity, training minimizes the Min-of-N (MoN) loss:

$\min_{\Theta} \sum_k \min_{1 \le j \le n} \left\{ d\left(G(I_k, r_j; \Theta), S_k^{gt}\right) \right\}$

where for each training instance $k$ , $I_k$ is the input image, $S_k^{gt}$ is the ground truth 3D point cloud, $r_1, r_2, \dots, r_n \sim \mathcal{N}(0, \mathbf{I})$ are $n$ independent identically distributed latent noise samples, and $d(\cdot, \cdot)$ is a permutation-invariant point set distance ( $d_{CD}$ or $d_{EMD}$ ).

By computing the loss strictly on the single prediction among the $n$ generated candidates that is closest to the ground truth, gradient updates encourage each latent branch to specialize in distinct plausible shape modes. In practice, setting $n = 2$ (Mo2) suffices to capture shape variance across ambiguous views.
Knowl 5 — Conditional Variational Autoencoder Formulation for Point Set Sampling
model/method
An alternative framework for learning a conditional shape distribution $P(S|X)$ from an input 2D image $X$ is a Conditional Variational Autoencoder (CVAE):
1. Training Phase: Ground truth 3D shape $S$ is converted to a volumetric voxel representation $Y$ . A 3D convolutional encoder $Q(z|Y, X)$ infers the parameters of a conditional Gaussian latent distribution $q(z|Y, X) = \mathcal{N}(\mu(Y, X), \Sigma(Y, X))$ . A latent code $z$ is sampled via the reparameterization trick, and a point set decoder network $f(z, X)$ predicts point coordinates $S^{pred}$ . The network is trained with the joint objective:
$\mathcal{L}_{CVAE} = d(f(z, X), S) + D_{KL}\left(Q(z|Y, X) \parallel \mathcal{N}(0, \mathbf{I})\right)$

where $d(\cdot, \cdot)$ is Chamfer Distance or Earth Mover's Distance, and $D_{KL}$ is the Kullback-Leibler divergence to an isotropic Gaussian prior.
1. Inference Phase: The 3D volumetric encoder $Q$ is discarded. Multiple distinct 3D point clouds are sampled for a single test image $X$ by drawing $z \sim \mathcal{N}(0, \mathbf{I})$ and decoding $S^{pred} = f(z, X)$ .
Knowl 6 — Comparative Mean-Shape Characteristics of Chamfer and Earth Mover's Distances
theoretical result
When a network is trained under ground truth ambiguity to minimize expected loss $\mathbb{E}_{S \sim \mathcal{P}}[L(S^{pred}, S)]$ , the predicted "mean shape" reflects the structural properties of the chosen distance metric:
- Continuous Geometric Variations (e.g., smoothly varying object radius or translation): Chamfer Distance (CD) matches points independently, which produces blurry, diffuse, and "splashy" point distributions around regions of high variance. Earth Mover's Distance (EMD) enforces a global 1-to-1 bijection constraint across the entire point budget, causing the predicted point set to concentrate sharply at the geometric expectation of the continuous parameter.
- Categorical / Discrete Variations (e.g., discrete multi-modal attachments or part presence/absence): CD places a fraction of the point budget at each possible discrete structural location without geometric distortion of individual parts. In contrast, EMD cannot split points across separate spatial modes without incurring large transportation costs, leading to compact but severely distorted hybrid geometries that compromise between the categorical options.
Knowl 7 — Point Cloud to Voxel Post-Processing Conversion
algorithm
To evaluate point set predictions against volumetric 3D reconstruction baselines using volumetric Intersection-over-Union (IoU), predicted point sets $S = \{(x_i, y_i, z_i)\}_{i=1}^N$ are rasterized into a $32 \times 32 \times 32$ binary occupancy grid through the following pipeline:
Input: Predicted point set $S \in \mathbb{R}^{N \times 3}$ , input image $I$
Output: Binary voxel occupancy grid $V_{out} \in \{0, 1\}^{32 \times 32 \times 32}$
1. Initialize voxel grid $V_{occ} \in \mathbb{R}^{32 \times 32 \times 32}$ to zero.
2. Splat each point $p \in S$ as a unit cube $[1 \times 1 \times 1]$ into $V_{occ}$ using trilinear/bilinear interpolation, accumulating the fractional volume overlap to form a continuous occupancy representation.
3. Predict scalar object volume $v_{pred} \in \mathbb{R}$ from image $I$ using a volume estimation network to normalize for point density differences across object scales.
4. Replicate $v_{pred}$ to form a spatial volume prior tensor $V_{vol} \in \mathbb{R}^{32 \times 32 \times 32}$ .
5. Concatenate $V_{occ}$ and $V_{vol}$ along the channel dimension to form a 2-channel tensor $V_{in} \in \mathbb{R}^{32 \times 32 \times 32 \times 2}$ .
6. Pass $V_{in}$ through a 3D CNN consisting of 6 convolutional layers with $3 \times 3 \times 3$ kernels (trained with voxel IoU loss).
7. Threshold the output probabilities at $0.5$ to yield $V_{out}$ .
return $V_{out}$

Knowl 8 — Single-View 3D Reconstruction Performance on ShapeNet Benchmark

data/table

Comparison of single-view 3D reconstruction performance measured by volumetric Intersection-over-Union (IoU on a $32^3$ voxel grid) against the 3D-R2N2 baseline across 13 ShapeNet object categories. Point set predictions from the Point Set Generation Network are post-processed into $32^3$ voxel grids. Higher IoU indicates higher reconstruction fidelity.

Category	Ours (1 view)	3D-R2N2 (1 view)	3D-R2N2 (3 views)	3D-R2N2 (5 views)
plane	0.601	0.513	0.549	0.561
bench	0.550	0.421	0.502	0.527
cabinet	0.771	0.716	0.763	0.772
car	0.831	0.798	0.829	0.836
chair	0.544	0.466	0.533	0.550
monitor	0.552	0.468	0.545	0.565
lamp	0.462	0.381	0.415	0.421
speaker	0.737	0.662	0.708	0.717
firearm	0.604	0.544	0.593	0.600
couch	0.708	0.628	0.690	0.706
table	0.606	0.513	0.564	0.580
cellphone	0.749	0.661	0.732	0.754
watercraft	0.611	0.513	0.596	0.610
Mean	0.640	0.560	0.617	0.631

In single-view 3D reconstruction, the Point Set Generation Network outperforms 1-view 3D-R2N2 across all 13 categories (mean IoU 0.640 vs. 0.560). Furthermore, the single-view point set network outperforms 5-view 3D-R2N2 on 8 out of 13 categories as well as in category-wise mean IoU (0.640 vs. 0.631). In direct point distance evaluations (where 1 unit is defined as $1/10$ of the ground truth grid length), the point set network achieves a Chamfer Distance of 0.25 (vs. 0.71 for 3D-R2N2) and an Earth Mover's Distance of 0.80 (vs. 1.02 for 3D-R2N2).

Knowl 9 — Ablation Analysis of Predictor Network Architectures
empirical result
Evaluating predictor architectures on synthetic ShapeNet renderings demonstrates the distinct contributions of the deconvolution branch, fully connected branch, and hourglass topology (measured in CD and EMD error, where lower values are better):
- Vanilla CNN (Pure Fully Connected Predictor): Achieves $\text{CD} = 0.50$ and $\text{EMD} = 0.88$ .
- Two Branches (Combined Fully Connected and Deconvolution): Achieves $\text{CD} = 0.43$ and $\text{EMD} = 0.83$ .
- Hourglass Version (Stacked Multi-Scale Predictor): Achieves $\text{CD} = 0.41$ and $\text{EMD} = 0.79$ .
Visualization of isolated predictor outputs reveals that the deconvolution branch forms a continuous 2D manifold wrapping the object's bulk volume, whereas the fully connected branch predicts unorganized, high-frequency structural elements (such as thin appendages, gun barrels, and chair legs).
Knowl 10 — Failure Modes in Single-View Point Set Reconstruction
limitation
The point set generation framework exhibits two primary failure modes:
1. Out-of-Distribution and Novel Geometries: When presented with an object category unseen during training, the network cannot reconstruct the novel topology and instead produces a distorted compromise from familiar training priors (for example, attempting to reconstruct an unfamiliar aircraft-like shape as an airplane without wings).
2. Multi-Object Compositions: In the absence of an explicit bounding box detection, instance segmentation, or visual attention mechanism, scenes containing multiple compositional objects result in entangled and geometrically corrupted reconstructions.

Coverage note — Minor implementation details (e.g., Blinn-Phong shading rendering parameters with local lighting models on 220k CAD models and the 16-model manual human baseline GUI modeling setup) were omitted as they serve purely as supporting experimental mechanics.

References

1.J. Aloimonos. Shape from texture. Biological cybernetics, 58(5):345–360, 1988.
2.D. P. Bertsekas. A distributed asynchronous relaxation algorithm for the assignment problem. In Decision and Control, 1985 24th IEEE Conference on, pages 1703–1704. IEEE, 1985.
3.J. Carreira, S. Vicente, L. Agapito, and J. Batista. Lifting object detection datasets into 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7):1342–1355, 2016.
4.A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR], 2015.
5.C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese. 3d-r2n2: A unified approach for single and multiview 3d object reconstruction. arXiv preprint arXiv:1604.00449, 2016.
6.C. Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908, 2016.
7.D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in neural information processing systems, pages 2366–2374, 2014.
8.Y. Eldar, M. Lindenbaum, M. Porat, and Y. Y. Zeevi. The farthest point strategy for progressive image sampling. IEEE Transactions on Image Processing, 6(9):1305–1315, 1997.
9.D. F. Fouhey, A. Gupta, and M. Hebert. Data-driven 3D primitives for single image understanding. In ICCV, 2013.
10.J. Fuentes-Pacheco, J. Ruiz-Ascencio, and J. M. Rendón-Mancha. Visual simultaneous localization and mapping: a survey. Artificial Intelligence Review, 43(1):55–81, 2015.
11.K. Häming and G. Peters. The structure-from-motion reconstruction pipeline–a survey with focus on short image sequences. Kybernetika, 46(5):926–937, 2010.
12.D. Hoiem, A. A. Efros, and M. Hebert. Automatic photo pop-up. ACM transactions on graphics (TOG), 24(3):577–584, 2005.
13.B. K. Horn. Obtaining shape from shading information. In Shape from shading, pages 123–171. MIT press, 1989.
14.Q. Huang, H. Wang, and V. Koltun. Single-view reconstruction via joint analysis of image and shape collections. ACM Transactions on Graphics (TOG), 34(4):87, 2015.
15.A. Kar, S. Tulsiani, J. Carreira, and J. Malik. Category-specific object reconstruction from a single image. In CVPR, 2015.
16.D. Maturana and S. Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In IEEE/RSJ International Conference on Intelligent Robots and Systems, September 2015.
17.M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
18.A. Newell, K. Yang, and J. Deng. Stacked hourglass networks for human pose estimation. arXiv preprint arXiv:1603.06937, 2016.
19.D. J. Rezende, S. Eslami, S. Mohamed, P. Battaglia, M. Jaderberg, and N. Heess. Unsupervised learning of 3d structure from images. arXiv preprint arXiv:1607.00662, 2016.
20.Y. Rubner, C. Tomasi, and L. J. Guibas. The earth mover’s distance as a metric for image retrieval. International journal of computer vision, 40(2):99–121, 2000.
21.A. Saxena, M. Sun, and A. Y. Ng. Make3d: Learning 3d scene structure from a single still image. IEEE transactions on pattern analysis and machine intelligence, 31(5):824–840, 2009.
22.H. Su, Q. Huang, N. J. Mitra, Y. Li, and L. Guibas. Estimating image depth using shape collections. ACM Transactions on Graphics (TOG), 33(4):37, 2014.

Citation

MLA

Fan, H., et al. “A Point Set Generation Network for 3D Object Reconstruction from a Single Image”. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2463–71, https://doi.org/10.1109/CVPR.2017.264.

APA

Fan, H., Su, H., & Guibas, L. (2017). A Point Set Generation Network for 3D Object Reconstruction from a Single Image. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2463–2471. https://doi.org/10.1109/CVPR.2017.264

Chicago

Fan, H., H. Su, and L. Guibas. 2017. “A Point Set Generation Network for 3D Object Reconstruction from a Single Image”. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2463–71. https://doi.org/10.1109/CVPR.2017.264.

Harvard

Fan, H., Su, H. and Guibas, L. (2017) “A Point Set Generation Network for 3D Object Reconstruction from a Single Image”, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp. 2463–2471. Available at: https://doi.org/10.1109/CVPR.2017.264.

Vancouver

1. Fan H, Su H, Guibas L (2017) A Point Set Generation Network for 3D Object Reconstruction from a Single Image. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2463–2471

BibTeX

@inproceedings{Fan_2017, title={A Point Set Generation Network for 3D Object Reconstruction from a Single Image}, url={http://dx.doi.org/10.1109/CVPR.2017.264}, DOI={10.1109/cvpr.2017.264}, booktitle={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, publisher={IEEE}, author={Fan, Haoqiang and Su, Hao and Guibas, Leonidas}, year={2017}, month=July, pages={2463–2471} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Dual-Branch Point Set Generation Network Architecture

Knowl 2 — Chamfer Distance for Point Set Prediction Loss

Knowl 3 — Earth Mover's Distance for Point Set Prediction Loss

Knowl 4 — Min-of-N Loss for Modeling Ground Truth Ambiguity

Knowl 5 — Conditional Variational Autoencoder Formulation for Point Set Sampling

Knowl 6 — Comparative Mean-Shape Characteristics of Chamfer and Earth Mover's Distances

Knowl 7 — Point Cloud to Voxel Post-Processing Conversion