SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks

Xiang XuKarl D. D. WillisJoseph G. LambourneChin-Yi ChengPradeep Kumar JayaramanYasutaka Furukawa

article2022ICML176 citations

Presents an autoregressive generative framework that factorizes parametric CAD construction sequences into disentangled topological, geometric, and extrusion codebooks, enabling precise user control and structured design space exploration.

Listen

Computer-aided design (CAD) is essential for modern mechanical and conceptual engineering. However, creating parametric CAD models requires deep technical expertise, and traditional models often break when subjected to major structural or topological modifications. While artificial intelligence offers the potential to automate 3D modeling, existing deep learning tools struggle to produce valid, complex engineering shapes and fail to give designers fine-grained control over specific geometric and structural properties.

The article demonstrates and evaluates SkexGen, an autoregressive generative framework designed to produce high-quality CAD construction sequences. The system explicitly separates CAD parameters into distinct, manageable representations to allow precise user control during automated design exploration.

The researchers developed a two-branch neural network based on Transformer architectures that models standard sketch-and-extrude operations—the primary paradigm used in commercial CAD software. The framework encodes designs into three separate, discrete dictionaries of visual parts, termed codebooks: one for 2D curve connectivity (topology), one for coordinate locations (geometry), and one for 3D extrusion actions. Using the large-scale DeepCAD benchmark dataset consisting of tens of thousands of cleaned CAD subsequences, the authors evaluated generation quality, model diversity, and disentangled control against several state-of-the-art baselines.

The article established several key findings across generation fidelity, design validity, and control. SkexGen achieved a visual fidelity score of 18.56 on 2D sketch generation, representing an improvement of roughly 75% to 83% over existing generative baselines, largely because it avoided producing self-intersecting, invalid geometry. For full 3D CAD models, the system achieved a coverage rate of 83.6% and a 99.8% novelty rate, outperforming previous methods in producing valid, multi-step construction sequences with realistic symmetry and arcs. A classifier confirmed that the three distinct codebooks achieved a 99.8% rate of parameter separation, allowing users to modify shapes, mix attributes between reference models, or interpolate designs while independently fixing topology, geometry, or extrusion parameters.

These findings indicate that discrete, separated codebooks can substantially improve the reliability of automated engineering tools. By generating valid boundary representations rather than noisy point clouds or non-watertight geometries, this approach reduces the labor needed to clean up AI-generated designs. For engineering workflows, this enhances design space exploration and accelerates early-stage prototyping while maintaining the direct structural editability required by manufacturing standards.

Organizations evaluating automated CAD tools should consider adopting modular architectures that separate high-level structure from dimensional geometry. Next practical steps supported by the article include integrating conditional code selectors into designer workflows to assist with part variation and design interpolation. Future development should also explore integrating geometric constraint solvers and expanding generation speed.

Confidence in the reported improvements is high based on quantitative benchmarks, though users should note operational limitations. Autoregressive sampling makes sequence generation slower than non-autoregressive baselines (taking 90 seconds versus 15 seconds per 10,000 samples). Additionally, the system relies on a 6-bit grid discretization (64 by 64 resolution) and a fixed set of canonical 3D rotation steps, which limits precision for complex continuous curves and highly specialized orientations.

  • Paper: Zero-Shot Text-to-Image Generation, Aditya Ramesh et al. (2021). Its discrete image tokenization and autoregressive Transformer provide a foundational example of the sequence-generation design that SkexGen adapts to CAD construction operations.
  • Paper: Taming Transformers for High-Resolution Image Synthesis, Patrick Esser et al. (2020). Its VQGAN codebook and autoregressive Transformer clarify the discrete-latent generation pipeline underlying SkexGen’s codebook-conditioned CAD sequence modeling.
Cover for SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks

Abstract

We present SkexGen, a novel autoregressive generative model for computer-aided design (CAD) construction sequences containing sketch-and-extrude modeling operations. Our model utilizes distinct Transformer architectures to encode topological, geometric, and extrusion variations of construction sequences into disentangled codebooks. Autoregressive Transformer decoders generate CAD construction sequences sharing certain properties specified by the codebook vectors. Extensive experiments demonstrate that our disentangled codebook representation generates diverse and high-quality CAD models, enhances user control, and enables efficient exploration of the design space.

Citation

MLA
Xu, X., et al. “SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks”. International Conference on Machine Learning, vol. 162, 2022, pp. 24698–724, https://proceedings.mlr.press/v162/xu22k.html.
APA
Xu, X., Willis, K. D. D., Lambourne, J. G., Cheng, C.-Y., Jayaraman, P. K., & Furukawa, Y. (2022). SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks. International Conference on Machine Learning, 162, 24698–24724. https://proceedings.mlr.press/v162/xu22k.html
Chicago
Xu, X., K. D. D. Willis, J. G. Lambourne, C.-Y. Cheng, P. K. Jayaraman, and Y. Furukawa. 2022. “SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks”. International Conference on Machine Learning 162: 24698–724. https://proceedings.mlr.press/v162/xu22k.html.
Harvard
Xu, X. et al. (2022) “SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks”, International Conference on Machine Learning. PMLR, pp. 24698–24724. Available at: https://proceedings.mlr.press/v162/xu22k.html.
Vancouver
1. Xu X, Willis KDD, Lambourne JG, Cheng C-Y, Jayaraman PK, Furukawa Y (2022) SkexGen: Autoregressive Generation of CAD Construction Sequences with Disentangled Codebooks. In: International Conference on Machine Learning. PMLR, pp 24698–24724

BibTeX

@InProceedings{pmlr-v162-xu22k,
  title = 	 {{S}kex{G}en: Autoregressive Generation of {CAD} Construction Sequences with Disentangled Codebooks},
  author =       {Xu, Xiang and Willis, Karl D.D. and Lambourne, Joseph G and Cheng, Chin-Yi and Jayaraman, Pradeep Kumar and Furukawa, Yasutaka},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {24698--24724},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/xu22k/xu22k.pdf},
  url = 	 {https://proceedings.mlr.press/v162/xu22k.html},
  abstract = 	 {We present SkexGen, a novel autoregressive generative model for computer-aided design (CAD) construction sequences containing sketch-and-extrude modeling operations. Our model utilizes distinct Transformer architectures to encode topological, geometric, and extrusion variations of construction sequences into disentangled codebooks. Autoregressive Transformer decoders generate CAD construction sequences sharing certain properties specified by the codebook vectors. Extensive experiments demonstrate that our disentangled codebook representation generates diverse and high-quality CAD models, enhances user control, and enables efficient exploration of the design space. The code is available at https://samxuxiang.github.io/skexgen.}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: https://creativecommons.org/licenses/by/4.0/