Back

Learning Implicit Fields for Generative Shape Modeling

Zhiqin ChenHao Zhang

article2018CVPR1,864 citations

Introduces IM-NET, an implicit field decoder that learns continuous 3D shape representations through point classification, enabling high-resolution shape generation, interpolation, and single-view reconstruction without the resolution constraints of conventional decoders.

Listen

Generating high-quality 3D digital shapes using deep learning is critical for computer vision, visual effects, and design workflows. However, existing methods relying on voxel grids, point clouds, or deformable surface patches face major limitations. Voxel models demand prohibitive memory at high resolutions and yield stepped, unnatural boundaries; point clouds lack cohesive surfaces; and patch deformation approaches struggle with fixed topologies, tears, and overlapping seams. The article addresses these core challenges by developing an implicit field decoder, termed IM-NET, designed to generate continuous, high-fidelity 3D shape surfaces while overcoming memory and topological constraints.

The article set out to evaluate whether an implicit field representation—trained as a binary classifier assigning inside/outside status to spatial coordinates—can outperform standard decoders in shape auto-encoding, generative modeling, shape interpolation, and single-view 3D reconstruction. The research tested IM-NET across benchmarks such as ShapeNet Core 3D categories (including airplanes, cars, chairs, rifles, and tables) and 2D MNIST handwritten digits. The method feeds shape feature vectors concatenated with point coordinates into a multi-layer neural network, evaluating reconstruction fidelity and visual similarity using metrics such as the Light Field Descriptor, Chamfer Distance, and Intersection over Union.

The findings demonstrate that IM-NET consistently produces superior visual quality, smooth continuous surfaces, and coherent geometry. In 3D generative modeling, IM-GAN achieved better visual diversity and lower shape distance metrics compared to leading baselines like 3DGAN and PC-GAN, while avoiding mode collapse and point-density limitations. In shape interpolation, IM-NET seamlessly handled geometric shifts and complex topological transitions—such as holes appearing or disappearing—without the fracturing common in convolutional networks or the seam artifacts in patch-based methods. For single-view 3D reconstruction, IM-NET generated clean, unified meshes at high sampling resolutions (up to 512 cubed) without being constrained by the lower voxel resolution of the training inputs. Although standard point-wise metrics like Chamfer Distance and Intersection over Union favored voxel networks because of rigid spatial alignment, visual evaluations and Light Field Descriptor scores confirmed that IM-NET captures superior boundary sharpness and human-perceptible quality.

These results imply that switching from discrete voxel generation to continuous implicit field modeling provides a scalable, memory-efficient alternative for 3D asset synthesis. The decoupled, coordinate-based approach enables high-resolution mesh extraction via Marching Cubes, eliminating the need for expensive high-resolution 3D convolutional architectures during training. Decision-makers and technical teams can leverage this architecture to enhance automated 3D reconstruction pipelines and creative design generation tools.

As next steps, the article recommends adopting implicit field decoders for tasks requiring complex shape topologies and visual realism. However, organizations must account for trade-offs: IM-NET incurs longer training and inference times because every spatial query point must pass through the network, making baseline convolutional autoencoders approximately 15 to 30 times faster. Further work should explore structural optimizations to accelerate inference, extend the decoder to predict additional attributes such as color, texture, and surface normals, and test multi-category generalizability, as the current framework was evaluated primarily on single-category models.

3d reconstructionbinary classificationdecodersgenerative shape modelingimplicit fieldshierarchical surface prediction
arXiv: 1812.02822czq142857/implicit-decoder
Cover for Learning Implicit Fields for Generative Shape Modeling

Abstract

We advocate the use of implicit fields for learning generative models of shapes and introduce an implicit field decoder, called IM-NET, for shape generation, aimed at improving the visual quality of the generated shapes. An implicit field assigns a value to each point in 3D space, so that a shape can be extracted as an iso-surface. IM-NET is trained to perform this assignment by means of a binary classifier. Specifically, it takes a point coordinate, along with a feature vector encoding a shape, and outputs a value which indicates whether the point is outside the shape or not. By replacing conventional decoders by our implicit decoder for representation learning (via IM-AE) and shape generation (via IM-GAN), we demonstrate superior results for tasks such as generative shape modeling, interpolation, and single-view 3D reconstruction, particularly in terms of visual quality. Code and supplementary material are available at this https URL.

Topics

3d reconstructionbinary classificationdecodersfeature vectorsgangenerative modelreconstructionrepresentation learningshape informationvisual qualities

Keywords

generative shape modelingimplicit fieldshierarchical surface predictionsingle-view reconstructionsWasserstein GANVariational Autoencodermesh reconstructionshape interpolation

Table of Contents

  • 1 Introduction
  • 2 Related work
  • 3 Implicit decoder and shape generation
  • 3.1 Data preparation
  • 3.2 Network structure of IM-NET
  • 3.3 Shape generation and other applications
  • 4 Results and evaluation
  • 4.1 Quality metrics
  • 4.2 Auto-encoding 3D shapes
  • 4.3 3D shape generation and interpolation
  • 4.4 2D shape generation and interpolation
  • 4.5 Single-view 3D reconstruction (SVR)
  • 5 Conclusion, limitation, and future work
  • References

Knowls

  1. Knowl 1 — IM-NET Implicit Field Decoder Architecture

    model/method

    IM-NET is an implicit field decoder designed to represent 2D or 3D continuous shapes by predicting spatial occupancy values. For a closed 3D shape, the continuous inside/outside field F(p)F(p) at a point coordinate p=(x,y,z)∈[0,1]3p = (x, y, z) ∈ [0, 1]^3 is defined as:

    F(p)={0if point p is outside the shape,1otherwise.F(p) = \begin{cases} 0 & \text{if point } p \text{ is outside the shape}, \\ 1 & \text{otherwise.} \end{cases}

    IM-NET parameterizes this field as a function fθ(p,z)f_\theta(p, z), where z∈R128z \in \mathbb{R}^{128} is a shape feature vector extracted by an encoder (such as a 3D CNN or ResNet) and pp is the point coordinate. The network architecture is a multi-layer perceptron (MLP) with skip connections:

    1. Input layer: Concatenates the 128-dimensional shape feature vector zz and the 3-dimensional coordinate pp into a 131-dimensional vector.
    2. Hidden layer 1: Fully-connected layer projecting 131 to 2048 units with LeakyReLU activation.
    3. Hidden layer 2: Concatenates the 2048-dimensional output of layer 1 with the original 128-dimensional feature vector zz (2176 units total) and projects to 1024 units with LeakyReLU.
    4. Hidden layer 3: Concatenates the 1024-dimensional output with zz (1152 units total) and projects to 512 units with LeakyReLU.
    5. Hidden layer 4: Concatenates the 512-dimensional output with zz (640 units total) and projects to 256 units with LeakyReLU.
    6. Hidden layer 5: Concatenates the 256-dimensional output with zz (384 units total) and projects to 128 units with LeakyReLU.
    7. Output layer: Fully-connected layer from 128 units to 1 unit with a Sigmoid activation function, outputting a continuous occupancy probability in [0,1][0, 1].

    At test time, the triangular mesh is reconstructed by querying fθ(p,z)f_\theta(p, z) across a regular grid in 3D space at an arbitrary target resolution and extracting the zero-isosurface via Marching Cubes at threshold k=0.5k = 0.5.

  2. Knowl 2 — Weighted Mean Squared Error Loss for Implicit Field Learning

    equation

    IM-NET is trained using a weighted mean squared error (MSE) loss between the predicted occupancy values and the ground-truth binary labels over a set of sampled spatial points. Given a set of sampled points SS, the loss function L(θ)L(\theta) parameterized by the network weights θ\theta is:

    L(θ)=∑p∈S∣fθ(p)−F(p)∣2⋅wp∑p∈SwpL(\theta) = \frac{\sum_{p \in S} |f_\theta(p) - F(p)|^2 \cdot w_p}{\sum_{p \in S} w_p}

    where:

    • p∈[0,1]3p \in [0, 1]^3 denotes a sampled spatial coordinate.
    • fθ(p)∈[0,1]f_\theta(p) \in [0, 1] is the scalar output predicted by IM-NET for point pp.
    • F(p)∈{0,1}F(p) \in \{0, 1\} is the ground-truth inside/outside occupancy indicator (0 for outside, 1 for inside).
    • wpw_p is a positive weight assigned to point pp, defined as the inverse of the local spatial sampling density near pp to compensate for non-uniform point sampling. Setting wp=1w_p = 1 for all points in practice focuses optimization heavily on boundaries while tolerating minor errors in distant void space.

  3. Knowl 3 — Progressive Multi-Resolution Point Sampling Strategy

    algorithm

    To train IM-NET without dense evaluation of O(n3)O(n^3) voxel coordinates, points are sampled near shape surfaces across progressive voxel resolutions (163,323,643,128316^3, 32^3, 64^3, 128^3) derived from high-resolution (2563256^3) voxelizations. Training advances coarse-to-fine while keeping the network architecture fixed.

    Input: High-resolution voxel shape, target resolution grid size n∈{16,32,64,128}n \in \{16, 32, 64, 128\}, point sample quotas N(16)=4096N(16)=4096, N(32)=8192N(32)=8192, N(64)=32768N(64)=32768, N(128)=131072N(128)=131072
    Output: Set of sampled point-value training pairs S={(p,F(p))}S = \{(p, F(p))\}
    if n=16n = 16 then
        Sample all 163=409616^3 = 4096 voxel center coordinates pp across the grid.
        Assign ground-truth occupancy F(p)∈{0,1}F(p) \in \{0, 1\} based on voxel state.
    else
        Initialize S=∅S = \emptyset
        // Approach 1: Surface boundary sampling
        Collect all grid points pp located within 3 voxels (in x,y,zx, y, z) from shape boundaries.
        if size of collected points ≤N(n)\le N(n) then
            Add collected points to SS.
            Randomly sample additional points uniformly from the volume until ∣S∣=N(n)|S| = N(n).
        else
            // Approach 2: Strided subsampling
            Collect grid points every two voxels along x,y,zx, y, z axes.
            if size of collected points ≤N(n)\le N(n) then
                Add collected points to SS.
                Randomly sample additional points until ∣S∣=N(n)|S| = N(n).
            else
                // Approach 3: Uniform random sampling
                Randomly sample N(n)N(n) points uniformly across the volume into SS.
            end if
        end if
        For each point p∈Sp \in S, evaluate ground-truth occupancy F(p)F(p).
    end if
    return SS

  4. Knowl 4 — Latent-GAN Generative Modeling of Implicit Fields (IM-GAN)

    model/method

    IM-GAN performs 3D generative shape modeling by decoupling shape representation learning from adversarial distribution learning using a two-stage pipeline:

    1. Autoencoder Pre-training (IM-AE): A 3D convolutional neural network (CNN) encoder compresses 64364^3 voxel shapes into 128-dimensional latent vectors z∈R128z \in \mathbb{R}^{128}. An implicit decoder (IM-NET) is jointly trained using progressive point sampling and weighted mean squared error to decode (p,z)(p, z) pairs into point occupancies.
    2. Latent Adversarial Learning (Latent-GAN): A Wasserstein Generative Adversarial Network with gradient penalty (WGAN-GP) is trained directly on the 128-dimensional latent space learned by the IM-AE encoder. Both the generator GG and discriminator DD consist of multi-layer perceptrons with two hidden fully-connected layers.
    3. Shape Synthesis and Extraction: To generate a new 3D shape, a noise vector is passed through GG to obtain a synthesized shape latent vector zsyn∈R128z_{\text{syn}} \in \mathbb{R}^{128}. The pre-trained IM-NET decoder evaluates fθ(p,zsyn)f_\theta(p, z_{\text{syn}}) across a regular 3D grid (such as 2563256^3 or 5123512^3), and Marching Cubes extracts the continuous zero-isosurface (at threshold k=0.5k = 0.5) to yield the output mesh.

  5. Knowl 5 — Single-View 3D Reconstruction via Latent Feature Regression (IM-SVR)

    model/method

    IM-SVR performs single-view 3D reconstruction by regressing 2D image observations into the learned latent space of a pre-trained implicit shape autoencoder:

    1. Autoencoder Pre-training: An implicit autoencoder (IM-AE) consisting of a 3D CNN encoder and an IM-NET decoder is pre-trained on 3D voxel models to learn a 128-dimensional shape latent space.
    2. 2D Encoder Optimization: The pre-trained IM-NET decoder parameters are frozen. A ResNet encoder is trained on 128×128128 \times 128 single-view grayscale images to output a 128-dimensional vector z^\hat{z}. Training minimizes the mean squared error between the predicted latent code z^\hat{z} and the ground-truth latent code zz generated by the pre-trained 3D IM-AE encoder on the corresponding 3D shape: LSVR=∥z^−z∥22L_{\text{SVR}} = \|\hat{z} - z\|_2^2
    3. Implicit Surface Reconstruction: Given an input test image, the trained ResNet predicts z^\hat{z}. The frozen IM-NET evaluates point occupancy values fθ(p,z^)f_\theta(p, \hat{z}) across an arbitrary resolution grid (e.g., 2563256^3), followed by iso-surface extraction via Marching Cubes.

  6. Knowl 6 — Quantitative Comparison of 3D Shape Auto-Encoding

    data/table

    Reconstruction performance of autoencoders trained on 64364^3 voxel shapes was evaluated across five ShapeNet categories (plane, car, chair, rifle, table) comparing a 3D CNN decoder (CNN-AE) against the implicit decoder (IM-AE). IM-AE outputs were evaluated when sampled at 64364^3 (IM64) and 2563256^3 (IM256) grid resolutions before applying Marching Cubes. Point clouds (2,048 points) were sampled from reconstructed meshes to compute symmetric Chamfer Distance (CD), and visual similarity was assessed using the Light Field Descriptor (LFD) based on multi-view silhouette projections.

    Metric Plane Car Chair Rifle Table
    CNN64-MSE (×103\times 10^3) 1.47 4.37 7.76 1.62 5.80
    IM64-MSE (×103\times 10^3) 2.14 4.99 11.43 1.91 10.67
    CNN64-IoU (×102\times 10^2) 86.07 90.73 74.22 78.37 84.67
    IM64-IoU (×102\times 10^2) 78.77 89.26 65.65 72.88 71.44
    CNN64-CD (×104\times 10^4) 3.51 5.31 7.34 3.48 7.45
    IM64-CD (×104\times 10^4) 4.22 5.28 8.96 3.78 12.05
    IM256-CD (×104\times 10^4) 4.23 5.44 9.05 3.77 11.54
    CNN64-LFD 3,375 1,323 2,555 3,515 1,824
    IM64-LFD 3,371 1,190 2,515 3,714 2,370
    IM256-LFD 3,236 1,147 2,453 3,602 2,201

    CNN-AE scores better on volumetric MSE and IoU because it directly optimizes voxel distributions on a discrete grid. However, IM-AE sampled at 2563256^3 achieves lower LFD scores on planes, cars, and chairs, reflecting smoother surface continuity and finer geometric features that volumetric metrics penalize due to minor global alignment shifts.

  7. Knowl 7 — Quantitative Evaluation of 3D Shape Generation

    data/table

    Generative performance of IM-GAN was evaluated against 3DGAN, point cloud GAN (PC-GAN), and a 3D CNN decoder GAN (CNN-GAN) across five ShapeNet object categories. Evaluation metrics are Coverage (COV-LFD, measuring sample diversity as percentage of test shapes matched; higher is better) and Minimum Matching Distance (MMD-LFD, measuring generative fidelity via nearest-neighbor LFD distance; lower is better).

    Metric Model Plane Car Chair Rifle Table Average
    COV-LFD (%) 3DGAN - 12.13 25.07 62.32 18.80 29.58∗^*
    PC-GAN 73.55 61.40 70.06 61.47 77.50 68.80
    CNN-GAN 69.22 73.00 77.73 61.26 83.73 72.99
    IM-GAN 70.33 69.33 75.44 65.26 86.43 73.36
    MMD-LFD 3DGAN - 1,993 4,365 4,476 5,208 4,010∗^*
    PC-GAN 3,737 1,360 3,143 3,891 2,822 2,991
    CNN-GAN 3,745 1,288 3,012 3,819 2,594 2,892
    IM-GAN 3,689 1,287 2,893 3,760 2,527 2,831

    (∗^*Average for 3DGAN excludes planes due to lack of a pre-trained model.)

    IM-GAN obtains the best overall average diversity (73.36% COV-LFD) and fidelity (2,831 MMD-LFD). 3DGAN exhibits severe mode collapse, while PC-GAN produces sparse point distributions lacking fine surface details. IM-GAN generates shapes with smoother, more coherent surfaces than CNN-GAN.

  8. Knowl 8 — Quantitative Evaluation of Single-View 3D Reconstruction

    data/table

    Single-view 3D reconstruction (SVR) using IM-SVR (sampled at 2563256^3) was benchmarked against Hierarchical Surface Prediction (HSP at 2563256^3), AtlasNet with 25 square patches (AtlasNet25, 28,900 vertices), and AtlasNet with one spherical patch (AtlasNetO, 7,446 vertices) on the first 100 test shapes of five ShapeNet categories. Evaluation was conducted using Light Field Descriptor (LFD; lower is better) and Chamfer Distance (CD×104\text{CD} \times 10^4; lower is better).

    LFD Metric Plane Car Chair Rifle Table
    HSP 6,307 2,009 4,255 6,360 3,765
    AtlasNet25 4,877 1,667 3,244 6,507 2,725
    AtlasNetO 5,208 1,751 4,124 6,117 3,909
    IM-SVR 4,743 1,658 3,321 5,067 2,918
    CD Metric (×104\times 10^4) Plane Car Chair Rifle Table
    HSP 16.33 12.27 28.96 15.60 27.41
    AtlasNet25 8.10 8.10 18.47 9.40 14.20
    AtlasNetO 11.37 8.25 25.35 10.67 21.67
    IM-SVR 9.74 9.68 22.09 10.57 18.97

    IM-SVR achieves the lowest LFD on planes, cars, and rifles while producing watertight closed surfaces. In contrast, AtlasNet25 outputs disjoint surface patches containing visible slits, overlaps, and foldovers; AtlasNetO is restricted to genus-0 spherical topology and cannot represent holes; and HSP yields overly smoothed surfaces lacking sharp details.

  9. Knowl 9 — Continuous Spatial Field Learning vs. Convolutional Intensity Modulation

    theoretical result

    CNN decoders and implicit field decoders (IM-NET) exhibit fundamentally different geometric behaviors during latent shape interpolation:

    1. CNN Decoders: A CNN operating on discretized pixel or voxel grids computes output activations via spatial convolution kernels that apply weighted averages over fixed grid windows. Because convolution operations lack explicit coordinate awareness, latent space interpolations cause existing structural components to fade out in intensity while new components fade in ("disappear and appear" artifact). When interpolating across unobserved spatial gaps, CNN decoders produce fractured or semi-transparent artifacts.
    2. Implicit Decoders (IM-NET): By providing continuous spatial coordinates pp alongside shape latent code zz directly to an MLP, IM-NET learns continuous decision boundaries fθ(p,z)=0.5f_\theta(p, z) = 0.5. Because the neural network defines an implicit function that is Lipschitz continuous over spatial coordinates, latent paths deform and translate shape boundaries continuously across space rather than modulating pixel/voxel intensities. This preserves clean surface boundaries, smoothly fills spatial gaps, and supports topology changes during morphing.

  10. Knowl 10 — Computational and Modeling Limitations of IM-NET

    limitation

    IM-NET incurs specific computational and geometric trade-offs:

    1. Training Time Complexity: Because the decoder must evaluate every sampled coordinate point individually, IM-AE is approximately 15×15\times slower to train than a 3D CNN-AE with progressive multi-resolution training, and up to 30×30\times slower without progressive training (requiring 1–2 days per category).
    2. Inference Latency: Generating an explicit 3D mesh requires evaluating the MLP across all n3n^3 coordinate points of a 3D grid prior to Marching Cubes iso-surface extraction. In contrast to CNN decoders that predict an entire volume in a single forward pass, IM-NET inference time scales directly with the sampling grid resolution n3n^3.
    3. Category Generalization and Topological Regulation: Models are trained per object category. Interpolation between highly dissimilar shapes or cross-category morphing lacks explicit topological constraints to ensure semantically plausible intermediate forms.
    4. Low-Frequency Metric Distortions: IM-NET optimizes point-wise classification boundaries, which can introduce low-frequency geometric distortions such as global thickening or thinning of parts (e.g., table legs), leading to higher volumetric MSE, lower IoU, and higher Chamfer Distance metrics despite superior visual smoothness.

Coverage note — The 2D MNIST digit generation experiment and its associated table were omitted to prioritize the core 3D generative, auto-encoding, and single-view reconstruction contributions.

References

  1. 1.Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas J Guibas. Learning representations and generative models for 3d point clouds. In International Conference on Machine Learning (ICML), 2018.
  2. 2.Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations (ICLR), 2017.
  3. 3.Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning (ICML), pages 214–223, 2017.
  4. 4.Matan Atzmon, Haggai Maron, and Yaron Lipman. Point convolutional neural networks by extension operators. ACM Trans. Graph., 37(4):71:1–71:12, July 2018.
  5. 5.Yoshua Bengio, Eric Laufer, Guillaume Alain, and Jason Yosinski. Deep generative stochastic networks trainable by backprop. In International Conference on Machine Learning (ICML), pages 226–234, 2014.
  6. 6.Yoshua Bengio, Grégoire Mesnil, Yann Dauphin, and Salah Rifai. Better mixing via deep representations. In International Conference on Machine Learning (ICML), pages 552–560, 2013.
  7. 7.Fausto Bernardini, Joshua Mittleman, Holly Rushmeier, Cláudio Silva, and Gabriel Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics (TVCG), 5(4):349–359, 1999.
  8. 8.Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR], Stanford University — Princeton University — Toyota Technological Institute at Chicago, 2015.
  9. 9.Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. On visual similarity based 3d model retrieval. In Computer graphics forum, volume 22, pages 223–232. Wiley Online Library, 2003.
  10. 10.Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV), 2016.
  11. 11.Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Transactions on Visualization and Computer Graphics (TVCG), 18(6):914–924, 2012.
  12. 12.Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning a deep convolutional network for image super-resolution. In Proceedings of European Conference on Computer Vision (ECCV), 2014.
  13. 13.Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 2, page 6, 2017.
  14. 14.Matheus Gadelha, Rui Wang, and Subhransu Maji. Multiresolution tree networks for 3d point cloud processing. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  15. 15.Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. Learning a predictable and generative vector representation for objects. In Proceedings of the European Conference on Computer Vision (ECCV), pages 484–499. Springer, 2016.
  16. 16.Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS), pages 2672–2680, 2014.
  17. 17.Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry. Atlasnet: A papier-mâché approach to learning 3d surface generation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  18. 18.Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems (NIPS), pages 5767–5777, 2017.
  19. 19.Christian Häne, Shubham Tulsiani, and Jitendra Malik. Hierarchical surface prediction for 3d object reconstruction. In Proceedings of the International Conference on 3D Vision (3DV). 2017.
  20. 20.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  21. 21.Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2):251–257, March 1991.
  22. 22.Qixing Huang, Hai Wang, and Vladlen Koltun. Single-view reconstruction via joint analysis of image and shape collections. ACM Transactions on Graphics (TOG), 34(4):87, 2015.
  23. 23.Zeng Huang, Tianye Li, Weikai Chen, Yajie Zhao, Jun Xing, Chloe LeGendre, Linjie Luo, Chongyang Ma, and Hao Li. Deep volumetric video from very sparse multi-view performance capture. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  24. 24.Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. International Conference on Learning Representations (ICLR), 2018.
  25. 25.Diederik P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039, 2018.
  26. 26.Diederik P Kingma and Max Welling. Auto-encoding variational bayes. International Conference on Learning Representations (ICLR), 2014.
  27. 27.Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  28. 28.Chunyuan Li, Hao Liu, Changyou Chen, Yunchen Pu, Liqun Chen, Ricardo Henao, and Lawrence Carin. Alice: Towards understanding adversarial learning for joint distribution matching. Advances in Neural Information Processing Systems (NIPS), 2017.
  29. 29.Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. Grass: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics (TOG), 36(4):52, 2017.
  30. 30.Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. PointCNN: Convolution on X-transformed points. 2018.
  31. 31.Yiyi Liao, Simon Donné, and Andreas Geiger. Deep marching cubes: Learning explicit surface representations. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  32. 32.Chen-Hsuan Lin, Chen Kong, and Simon Lucey. Learning efficient point cloud generation for dense 3d object reconstruction. In AAAI Conference on Artificial Intelligence (AAAI), 2018.
  33. 33.William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. SIGGRAPH Computer Graphics, 21(4):163–169, August 1987.
  34. 34.Daniel Cohen-Or Olga Sorkine and Sivan Toledo. High-pass quantization for mesh encoding. In Eurographics Sym. on Geometry Processing, 2015.
  35. 35.Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems (NIPS), 2017.
  36. 36.Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  37. 37.Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. International Conference on Learning Representations (ICLR), 2016.
  38. 38.Gernot Riegler, Ali Osman Ulusoy, Horst Bischof, and Andreas Geiger. Octnetfusion: Learning depth fusion from data. In Proceedings of the International Conference on 3D Vision (3DV), pages 57–66. IEEE, 2017.
  39. 39.Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. Neural Networks, 20(1):61–80, 2009.
  40. 40.Ayan Sinha, Jing Bai, and Karthik Ramani. Deep learning 3d shape surfaces using geometry images. In Proceedings of the European Conference on Computer Vision (ECCV), pages 223–240. Springer, 2016.
  41. 41.Ayan Sinha, Asim Unmesh, Qixing Huang, and Karthik Ramani. Surfnet: Generating 3d shape surfaces using deep residual networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 791–800, 2017.
  42. 42.Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015.
  43. 43.Hao Su, Qixing Huang, Niloy J Mitra, Yangyan Li, and Leonidas Guibas. Estimating image depth using shape collections. ACM Transactions on Graphics (TOG), 33(4):37, 2014.
  44. 44.Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), volume 2, page 8, 2017.
  45. 45.Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in Neural Information Processing Systems (NIPS), pages 4790–4798, 2016.
  46. 46.Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  47. 47.Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis. ACM Transactions on Graphics (SIGGRAPH), 36(4), 2017.
  48. 48.Peng-Shuai Wang, Chun-Yu Sun, Yang Liu, and Xin Tong. Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes. ACM Transactions on Graphics (SIGGRAPH Asia), 37(6), 2018.
  49. 49.Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In Advances in Neural Information Processing Systems (NIPS), pages 82–90, 2016.
  50. 50.Jiajun Wu, Chengkai Zhang, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, and Joshua B. Tenenbaum. Learning shape priors for single-view 3d completion and reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV), 2018.
  51. 51.Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud autoencoder via deep grid deformation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 3, 2018.
  52. 52.Kangxue Yin, Hui Huang, Daniel Cohen-Or, and Hao Zhang. P2P-NET: Bidirectional point displacement net for shape transform. ACM Trans. on Graph., 37(4):Article 152, 2018.
  53. 53.Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Renjiao Yi, and Hao Zhang. SCORES: Shape composition with recursive substructure priors. ACM Trans. on Graph., 37(6), 2018.

Citation

MLA
Chen, Z., and H. Zhang. “Learning Implicit Fields for Generative Shape Modeling”. arXiv, 2018, http://arxiv.org/abs/1812.02822v5.
APA
Chen, Z., & Zhang, H. (2018). Learning Implicit Fields for Generative Shape Modeling. arXiv. http://arxiv.org/abs/1812.02822v5
Chicago
Chen, Z., and H. Zhang. 2018. “Learning Implicit Fields for Generative Shape Modeling”. arXiv. http://arxiv.org/abs/1812.02822v5.
Harvard
Chen, Z. and Zhang, H. (2018) “Learning Implicit Fields for Generative Shape Modeling”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1812.02822v5.
Vancouver
1. Chen Z, Zhang H (2018) Learning Implicit Fields for Generative Shape Modeling. arXiv

BibTeX

@article{chen2018learning,
  title = {Learning Implicit Fields for Generative Shape Modeling},
  author = {Chen, Zhiqin and Zhang, Hao},
  year = {2018},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1812.02822v5},
  eprint = {1812.02822}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: IEEE