Back

Modeling Tabular data using Conditional GAN

Lei XuMaria SkoularidouAlfredo Cuesta-InfanteKalyan Veeramachaneni

article2019NeurIPS2,185 citations

Introduces TGAN, a conditional generative adversarial network designed to generate realistic synthetic tabular data by effectively modeling mixed data types, multimodal continuous distributions, and imbalanced categorical attributes.

deep learninggangenerative adversarial networksConditional GANtabular dataCTGAN model
arXiv: 1907.00503
  • Paper: Conditional Generative Adversarial Nets, Mehdi Mirza et al. (2014). This foundational work introduces conditional generative adversarial networks, providing the core algorithmic architecture that the source paper adapts to model tabular data.
  • Paper: TabNet: Attentive Interpretable Tabular Learning, Sercan Ö. Arik et al. (2019). Reading this paper first establishes essential context regarding modern deep learning approaches for tabular data before exploring the source model.

No sufficiently relevant recommendations were found.

Cover for Modeling Tabular data using Conditional GAN

Abstract

Modeling the probability distribution of rows in tabular data and generating realistic synthetic data is a non-trivial task. Tabular data usually contains a mix of discrete and continuous columns. Continuous columns may have multiple modes whereas discrete columns are sometimes imbalanced making the modeling difficult. Existing statistical and deep neural network models fail to properly model this type of data. We design TGAN, which uses a conditional generative adversarial network to address these challenges. To aid in a fair and thorough comparison, we design a benchmark with 7 simulated and 8 real datasets and several Bayesian network baselines. TGAN outperforms Bayesian methods on most of the real datasets whereas other deep learning methods could not.

Topics

deep learninggangenerative adversarial networksprobability distributionssynthetic datareal data setsconvolutional neural networksfeedforward neural networkssub-arraysbayesian inference

Keywords

Conditional GANtabular dataCTGAN modelvariational Gaussian mixture modelconditional generatorvariational autoencodersynthetic data generator

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Challenges with GANs in Tabular Data Generation Task
  • 4 CTGAN Model
  • 4.1 Notations
  • 4.2 Mode-specific Normalization
  • 4.3 Conditional Generator and Training-by-Sampling
  • 4.4 Network Structure
  • 4.5 TVAE Model
  • 5 Benchmarking Synthetic Data Generation Algorithms
  • 5.1 Baselines and Datasets
  • 5.2 Evaluation Metrics and Framework
  • 5.3 Benchmarking Results
  • 5.4 Ablation Study
  • 6 Conclusion
  • References
  • 7 Dataset Details

Knowls

  1. Knowl 1 — Mode-Specific Normalization for Continuous Tabular Features

    model/method

    Continuous variables in tabular datasets often exhibit complex, multi-modal, and non-Gaussian distributions. Standard min-max normalization to [−1,1][-1, 1] can lead to vanishing gradients in neural networks. Mode-specific normalization represents each continuous value using a one-hot vector indicating its assigned Gaussian mixture mode and a scalar indicating its relative position within that mode.

    For each continuous column CiC_i with observations ci,jc_{i,j}:

    1. A Variational Gaussian Mixture Model (VGM) is fitted to CiC_i to estimate the number of modes mim_i and the mixture parameters: PCi(ci,j)=∑k=1miμkN(ci,j;ηk,ϕk)P_{C_i}(c_{i,j}) = \sum_{k=1}^{m_i} \mu_k \mathcal{N}(c_{i,j}; \eta_k, \phi_k) where μk\mu_k, ηk\eta_k, and ϕk\phi_k are the mixture weight, mean, and standard deviation of mode kk, respectively.
    2. For each scalar value ci,jc_{i,j}, the probability density of it originating from mode k∈{1,…,mi}k \in \{1, \dots, m_i\} is computed as ρk=μkN(ci,j;ηk,ϕk)\rho_k = \mu_k \mathcal{N}(c_{i,j}; \eta_k, \phi_k).
    3. A single mode kk is sampled according to the normalized probability distribution [ρ1,…,ρmi][\rho_1, \dots, \rho_{m_i}].
    4. The mode indicator is encoded as a one-hot vector βi,j∈{0,1}mi\beta_{i,j} \in \{0, 1\}^{m_i} with βi,j(k)=1\beta_{i,j}^{(k)} = 1 and 0 elsewhere.
    5. The continuous value within the selected mode kk is normalized to a scalar: αi,j=ci,j−ηk4ϕk\alpha_{i,j} = \frac{c_{i,j} - \eta_k}{4\phi_k} which maps approximately 99.99% of values within 4σ4\sigma of the mode mean to [−1,1][-1, 1].

    For a dataset with NcN_c continuous columns and NdN_d discrete columns (where each discrete value is one-hot encoded as di,jd_{i,j}), the complete representation of row jj becomes the vector concatenation: rj=α1,j⊕β1,j⊕⋯⊕αNc,j⊕βNc,j⊕d1,j⊕⋯⊕dNd,jr_j = \alpha_{1,j} \oplus \beta_{1,j} \oplus \dots \oplus \alpha_{N_c,j} \oplus \beta_{N_c,j} \oplus d_{1,j} \oplus \dots \oplus d_{N_d,j}

  2. Knowl 2 — Conditional Vector and Training-by-Sampling in CTGAN

    model/method

    To address severe category imbalance and prevent mode collapse across discrete attributes without distorting the learned joint distribution, CTGAN introduces a conditional vector representation, a modified generator loss, and a training-by-sampling mechanism.

    Let D1,…,DNdD_1, \dots, D_{N_d} denote the NdN_d discrete columns, where each discrete value is one-hot encoded as di∈{0,1}∣Di∣d_i \in \{0, 1\}^{|D_i|}. A condition is specified as a selected discrete column Di∗D_{i^*} having category k∗k^*, denoted (Di∗=k∗)(D_{i^*} = k^*).

    1. Conditional Vector: The condition is represented by a concatenated mask vector: cond=m1⊕m2⊕⋯⊕mNd\text{cond} = m_1 \oplus m_2 \oplus \dots \oplus m_{N_d} where each sub-vector mi=[mi(1),…,mi(∣Di∣)]∈{0,1}∣Di∣m_i = [m_i^{(1)}, \dots, m_i^{(|D_i|)}] \in \{0, 1\}^{|D_i|} is defined as: mi(k)={1if i=i∗ and k=k∗0otherwisem_i^{(k)} = \begin{cases} 1 & \text{if } i = i^* \text{ and } k = k^* \\ 0 & \text{otherwise} \end{cases}

    2. Training-by-Sampling: At each training step:

    • A discrete column index i∗∈{1,…,Nd}i^* \in \{1, \dots, N_d\} is selected uniformly at random.
    • A discrete probability mass function (PMF) over the categories k∈Di∗k \in D_{i^*} is constructed such that the probability mass of each category is proportional to the logarithm of its frequency in that column.
    • A category k∗k^* is randomly sampled from this log-frequency PMF.
    • The condition vector cond\text{cond} is created with mi∗(k∗)=1m_{i^*}^{(k^*)} = 1.
    • A real training row rr is sampled uniformly from the subset of training data satisfying the condition, r∼Uniform(Ttrain∣Di∗=k∗)r \sim \text{Uniform}(T_{\text{train}} \mid D_{i^*} = k^*).
    1. Generator Cross-Entropy Loss: To enforce that the generator's output discrete feature d^i∗\hat{d}_{i^*} matches the conditioned category mi∗m_{i^*}, the cross-entropy loss between mi∗m_{i^*} and d^i∗\hat{d}_{i^*} is added to the generator objective: LCE=CrossEntropy(d^i∗,mi∗)\mathcal{L}_{\text{CE}} = \text{CrossEntropy}(\hat{d}_{i^*}, m_{i^*})

    This conditional mechanism ensures that the generator learns the conditional distribution PG(row∣Di∗=k∗)=P(row∣Di∗=k∗)P_G(\text{row} \mid D_{i^*} = k^*) = P(\text{row} \mid D_{i^*} = k^*), enabling exact reconstruction of the original joint distribution at test time via marginalization: P(row)=∑k∈Di∗PG(row∣Di∗=k)P(Di∗=k)P(\text{row}) = \sum_{k \in D_{i^*}} P_G(\text{row} \mid D_{i^*} = k) P(D_{i^*} = k)

  3. Knowl 3 — CTGAN Network Architectures and Loss Formulation

    model/method

    CTGAN models tabular row distributions using fully connected generator and critic networks trained with the Wasserstein GAN with Gradient Penalty (WGAN-GP) objective and PacGAN packing to prevent mode collapse.

    Conditional Generator Network: Given a latent noise vector z∼N(0,I)z \sim \mathcal{N}(0, I) and conditional vector cond\text{cond}, the generator produces continuous scalars α^i\hat{\alpha}_i, mode indicators β^i\hat{\beta}_i, and discrete vectors d^i\hat{d}_i:

    h_0 &= z \oplus \text{cond} \\ h_1 &= h_0 \oplus \text{ReLU}(\text{BN}(\text{FC}_{|\text{cond}|+|z| \to 256}(h_0))) \\ h_2 &= h_1 \oplus \text{ReLU}(\text{BN}(\text{FC}_{|\text{cond}|+|z|+256 \to 256}(h_1))) \\ \hat{\alpha}_i &= \tanh(\text{FC}_{|\text{cond}|+|z|+512 \to 1}(h_2)), \quad 1 \le i \le N_c \\ \hat{\beta}_i &= \text{gumbel}_{0.2}(\text{FC}_{|\text{cond}|+|z|+512 \to m_i}(h_2)), \quad 1 \le i \le N_c \\ \hat{d}_i &= \text{gumbel}_{0.2}(\text{FC}_{|\text{cond}|+|z|+512 \to |D_i|}(h_2)), \quad 1 \le i \le N_d \end{aligned}$$ where $\text{BN}$ is batch normalization, $\text{FC}_{u \to v}$ is a linear layer mapping from dimension $u$ to $v$, and $\text{gumbel}_{0.2}$ denotes the Gumbel-Softmax activation with temperature parameter $\tau = 0.2$. **Critic Network with PacGAN (Pac Size 10):** The critic processes a pack of 10 concatenated (row, condition) pairs $(r_1 \oplus \dots \oplus r_{10} \oplus \text{cond}_1 \oplus \dots \oplus \text{cond}_{10})$: $$\begin{aligned} h_0 &= r_1 \oplus \dots \oplus r_{10} \oplus \text{cond}_1 \oplus \dots \oplus \text{cond}_{10} \\ h_1 &= \text{drop}(\text{leaky}_{0.2}(\text{FC}_{10|r|+10|\text{cond}| \to 256}(h_0))) \\ h_2 &= \text{drop}(\text{leaky}_{0.2}(\text{FC}_{256 \to 256}(h_1))) \\ C(\cdot) &= \text{FC}_{256 \to 1}(h_2) \end{aligned}$$ where $|r| = \sum_{i=1}^{N_c} (1 + m_i) + \sum_{i=1}^{N_d} |D_i|$, $\text{leaky}_{0.2}$ is LeakyReLU with negative slope $\gamma = 0.2$, and $\text{drop}$ applies dropout. **Optimization Objectives:** For a batch of $m$ rows grouped into $K = m / 10$ packs, the critic parameters $\Phi_C$ are optimized via WGAN-GP: $$\mathcal{L}_C = \frac{1}{K} \sum_{k=1}^K C(\hat{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})}) - \frac{1}{K} \sum_{k=1}^K C(r_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})}) + 10 \cdot \frac{1}{K} \sum_{k=1}^K \left(\|\nabla_{\tilde{r}_k^{(\text{pac})}} C(\tilde{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})})\|_2 - 1\right)^2$$ where $\tilde{r}_k^{(\text{pac})} = \rho_k \hat{r}_k^{(\text{pac})} + (1 - \rho_k) r_k^{(\text{pac})}$ with $\rho_k \sim \text{Uniform}(0, 1)$. The generator parameters $\Phi_G$ minimize: $$\mathcal{L}_G = -\frac{1}{K} \sum_{k=1}^K C(\hat{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})}) + \frac{1}{m} \sum_{j=1}^m \text{CrossEntropy}(\hat{d}_{i^*,j}, m_{i^*})$$ Both networks are updated using Adam with a learning rate of $2 \times 10^{-4}$ and batch size $m = 500$ for 300 epochs.

  4. Knowl 4 — CTGAN Training Step Algorithm

    algorithm

    The training step procedure for CTGAN updates the conditional generator parameters ΦG\Phi_G and critic parameters ΦC\Phi_C given a batch of training data TtrainT_{\text{train}}, batch size mm, and pac size pac=10\text{pac} = 10.

    Input: Training data TtrainT_{\text{train}}, generator parameters ΦG\Phi_G, critic parameters ΦC\Phi_C, batch size mm, pac size pac\text{pac}
    Output: Updated generator parameters ΦG\Phi_G and critic parameters ΦC\Phi_C
    for j=1j = 1 to mm do
        Randomly select discrete column index i∗∈{1,…,Nd}i^* \in \{1, \dots, N_d\} with equal probability
        Construct PMF across categories of Di∗D_{i^*} proportional to the logarithm of their frequencies in TtrainT_{\text{train}}
        Sample category k∗∼PMFk^* \sim \text{PMF}
        Create masks m1,…,mNdm_1, \dots, m_{N_d} where mi∗(k∗)=1m_{i^*}^{(k^*)} = 1 and all other components are 0
        condj←m1⊕⋯⊕mNd\text{cond}_j \leftarrow m_1 \oplus \dots \oplus m_{N_d}
        Sample latent vector zj∼N(0,I)z_j \sim \mathcal{N}(0, I)
        r^j←Generator(zj,condj)\hat{r}_j \leftarrow \text{Generator}(z_j, \text{cond}_j)
        Sample real row rj∼Uniform(Ttrain∣Di∗=k∗)r_j \sim \text{Uniform}(T_{\text{train}} \mid D_{i^*} = k^*)
    end for
    for k=1k = 1 to m/pacm / \text{pac} do
        condk(pac)←cond(k−1)pac+1⊕⋯⊕condk⋅pac\text{cond}_k^{(\text{pac})} \leftarrow \text{cond}_{(k-1)\text{pac}+1} \oplus \dots \oplus \text{cond}_{k \cdot \text{pac}}
        r^k(pac)←r^(k−1)pac+1⊕⋯⊕r^k⋅pac\hat{r}_k^{(\text{pac})} \leftarrow \hat{r}_{(k-1)\text{pac}+1} \oplus \dots \oplus \hat{r}_{k \cdot \text{pac}}
        rk(pac)←r(k−1)pac+1⊕⋯⊕rk⋅pacr_k^{(\text{pac})} \leftarrow r_{(k-1)\text{pac}+1} \oplus \dots \oplus r_{k \cdot \text{pac}}
    end for
    LC←1m/pac∑k=1m/pacCritic(r^k(pac),condk(pac))−1m/pac∑k=1m/pacCritic(rk(pac),condk(pac))L_C \leftarrow \frac{1}{m/\text{pac}} \sum_{k=1}^{m/\text{pac}} \text{Critic}(\hat{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})}) - \frac{1}{m/\text{pac}} \sum_{k=1}^{m/\text{pac}} \text{Critic}(r_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})})
    Sample ρ1,…,ρm/pac∼Uniform(0,1)\rho_1, \dots, \rho_{m/\text{pac}} \sim \text{Uniform}(0, 1)
    for k=1k = 1 to m/pacm / \text{pac} do
        r~k(pac)←ρkr^k(pac)+(1−ρk)rk(pac)\tilde{r}_k^{(\text{pac})} \leftarrow \rho_k \hat{r}_k^{(\text{pac})} + (1 - \rho_k) r_k^{(\text{pac})}
    end for
    LGP←1m/pac∑k=1m/pac(∥∇r~k(pac)Critic(r~k(pac),condk(pac))∥2−1)2L_{\text{GP}} \leftarrow \frac{1}{m/\text{pac}} \sum_{k=1}^{m/\text{pac}} (\|\nabla_{\tilde{r}_k^{(\text{pac})}} \text{Critic}(\tilde{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})})\|_2 - 1)^2
    ΦC←ΦC−0.0002×Adam(∇ΦC(LC+10LGP))\Phi_C \leftarrow \Phi_C - 0.0002 \times \text{Adam}(\nabla_{\Phi_C} (L_C + 10 L_{\text{GP}}))
    Regenerate fake rows r^j←Generator(zj,condj)\hat{r}_j \leftarrow \text{Generator}(z_j, \text{cond}_j) for j=1,…,mj = 1, \dots, m and pack them into r^k(pac)\hat{r}_k^{(\text{pac})}
    LG←−1m/pac∑k=1m/pacCritic(r^k(pac),condk(pac))+1m∑j=1mCrossEntropy(d^i∗,j,mi∗)L_G \leftarrow -\frac{1}{m/\text{pac}} \sum_{k=1}^{m/\text{pac}} \text{Critic}(\hat{r}_k^{(\text{pac})}, \text{cond}_k^{(\text{pac})}) + \frac{1}{m} \sum_{j=1}^m \text{CrossEntropy}(\hat{d}_{i^*, j}, m_{i^*})
    ΦG←ΦG−0.0002×Adam(∇ΦGLG)\Phi_G \leftarrow \Phi_G - 0.0002 \times \text{Adam}(\nabla_{\Phi_G} L_G)

  5. Knowl 5 — Tabular Variational Autoencoder (TVAE) Architecture and Loss

    model/method

    TVAE adapts the Variational Autoencoder (VAE) framework to mixed-type tabular data by applying mode-specific normalization for continuous columns and training via the evidence lower-bound (ELBO) objective with factorized reconstruction distributions.

    Generative Network pθ(rj∣zj)p_\theta(r_j \mid z_j): Given a latent sample zj∈R128z_j \in \mathbb{R}^{128}, the decoder produces parameters for NcN_c continuous columns (represented as scalar value αi,j\alpha_{i,j} and mode βi,j\beta_{i,j}) and NdN_d discrete columns di,jd_{i,j}:

    h_1 &= \text{ReLU}(\text{FC}_{128 \to 128}(z_j)) \\ h_2 &= \text{ReLU}(\text{FC}_{128 \to 128}(h_1)) \\ \bar{\alpha}_{i,j} &= \tanh(\text{FC}_{128 \to 1}(h_2)), \quad 1 \le i \le N_c \\ \hat{\alpha}_{i,j} &\sim \mathcal{N}(\bar{\alpha}_{i,j}, \delta_i), \quad 1 \le i \le N_c \\ \hat{\beta}_{i,j} &\sim \text{softmax}(\text{FC}_{128 \to m_i}(h_2)), \quad 1 \le i \le N_c \\ \hat{d}_{i,j} &\sim \text{softmax}(\text{FC}_{128 \to |D_i|}(h_2)), \quad 1 \le i \le N_d \end{aligned}$$ where $\delta_i$ is a trainable variance parameter per continuous column $C_i$. The joint likelihood of a row $r_j$ is factorized as: $$p_\theta(r_j \mid z_j) = \prod_{i=1}^{N_c} P(\hat{\alpha}_{i,j} = \alpha_{i,j}) \prod_{i=1}^{N_c} P(\hat{\beta}_{i,j} = \beta_{i,j}) \prod_{i=1}^{N_d} P(\hat{d}_{i,j} = d_{i,j})$$ **Inference Network $q_\phi(z_j \mid r_j)$:** $$\begin{aligned} h_1 &= \text{ReLU}(\text{FC}_{|r_j| \to 128}(r_j)) \\ h_2 &= \text{ReLU}(\text{FC}_{128 \to 128}(h_1)) \\ \mu &= \text{FC}_{128 \to 128}(h_2) \\ \sigma &= \exp\left(\frac{1}{2} \text{FC}_{128 \to 128}(h_2)\right) \\ q_\phi(z_j \mid r_j) &\sim \mathcal{N}(\mu, \sigma I) \end{aligned}$$ TVAE parameters $(\theta, \phi, \{\delta_i\})$ are trained end-to-end using the Adam optimizer with a learning rate of $10^{-3}$ to maximize the ELBO.

  6. Knowl 6 — Likelihood Fitness and Machine Learning Efficacy Evaluation Metrics

    model/method

    To evaluate tabular synthetic data generators across synthetic and real domains, an evaluation framework is defined along two axes:

    1. Likelihood Fitness (Evaluated on Simulated Data): When data is sampled from a known ground-truth oracle SS (such as a Gaussian mixture or Bayesian network):
    • Synthetic Likelihood Lsyn\mathcal{L}_{\text{syn}}: Evaluates the log-likelihood of synthetic data TsynT_{\text{syn}} under the true oracle SS. Because Lsyn\mathcal{L}_{\text{syn}} can unfairly favor overfitted models, it is complemented by Ltest\mathcal{L}_{\text{test}}.
    • Test Likelihood Ltest\mathcal{L}_{\text{test}}: A re-parameterized oracle S′S' having the exact same structure as SS (e.g., the same Bayesian network DAG or same number of Gaussian components) is fit from scratch using only TsynT_{\text{syn}}. Ltest\mathcal{L}_{\text{test}} is the log-likelihood of the held-out real test set TtestT_{\text{test}} evaluated under S′S'. Ltest\mathcal{L}_{\text{test}} effectively detects mode collapse in generated data.
    1. Machine Learning Efficacy (Evaluated on Real Data): For real-world tabular data where the true data-generating distribution is inaccessible:
    • Predictive machine learning models (such as Decision Trees, Logistic Regression, AdaBoost, and MLPs) are trained exclusively on TsynT_{\text{syn}} and evaluated on the actual test set TtestT_{\text{test}}.
    • Classification tasks are evaluated using Macro F1 score and Accuracy; regression tasks are evaluated using R2R^2.
    • The metric reported is the average score across all selected prediction models, benchmarked against the "Identity" baseline where models are trained on the real training set TtrainT_{\text{train}}.

  7. Knowl 7 — Benchmark Evaluation of Tabular Generative Models on Simulated and Real Datasets

    empirical result

    CTGAN and TVAE were evaluated against two Bayesian network baselines (CLBN, PrivBN) and three tabular GAN baselines (MedGAN, VeeGAN, TableGAN) across 7 simulated datasets (Grid, GridR, Ring, Alarm, Child, Asia, Insurance) and 8 real datasets (adult, census, covertype, intrusion, credit, mnist12, mnist28, news). All models were trained for 300 epochs with batch size 500.

    Method GM Sim. BN Sim. Real
    Lsyn\mathcal{L}_{\text{syn}} Ltest\mathcal{L}_{\text{test}} Lsyn\mathcal{L}_{\text{syn}} Ltest\mathcal{L}_{\text{test}} clf (F1) reg (R2R^2)
    Identity -2.61 -2.61 -9.33 -9.36 0.743 0.14
    CLBN -3.06 -7.31 -10.66 -9.92 0.382 -6.28
    PrivBN -3.38 -12.42 -12.97 -10.90 0.225 -4.49
    MedGAN -7.27 -60.03 -11.14 -12.15 0.137 -8.80
    VEEGAN -10.06 -4.22 -15.40 -13.86 0.143 -6.5e6
    TableGAN -8.24 -4.12 -11.84 -10.47 0.162 -3.09
    TVAE -2.65 -5.42 -6.76 -9.59 0.519 -0.20
    CTGAN -5.72 -3.40 -11.67 -10.60 0.469 -0.43

    Key results:

    1. On real datasets, CTGAN achieves an average classification F1 of 0.469 and regression R2R^2 of -0.43, outperforming Bayesian networks (CLBN F1: 0.382; PrivBN F1: 0.225) and all GAN baselines (MedGAN F1: 0.137; VeeGAN F1: 0.143; TableGAN F1: 0.162). TVAE achieves the highest overall scores (F1: 0.519, R2R^2: -0.20).
    2. On real datasets, CTGAN outperforms CLBN on 7 out of 8 datasets and outperforms PrivBN on 8 out of 8 datasets.
    3. Bayesian networks struggle on continuous columns due to mandatory discretization, resulting in poor regression performance (−6.28-6.28 and −4.49-4.49 R2R^2). Prior GAN baselines suffered severe mode collapse.

  8. Knowl 8 — Ablation Study on CTGAN Normalization, Sampling, and Architectural Components

    empirical result

    An ablation study on real classification datasets (excluding MNIST) measures the impact of mode-specific normalization, conditional generation, training-by-sampling, and adversarial architectures relative to the full CTGAN model:

    Mode-specific Normalization Generator Network Architecture
    Ablation GMM5 GMM10 MinMax w/o S. w/o C. GAN WGANGP
    Perf. Change -4.1% -8.6% -25.7% -17.8% -36.5% -6.5% +1.75%

    (GAN + PacGAN without WGAN-GP resulted in a −5.2%-5.2\% change.)

    Key takeaways from the ablation analysis:

    1. Mode-Specific Normalization: Replacing the Variational Gaussian Mixture Model (VGM) with fixed-component GMMs degrades performance by −4.1%-4.1\% (5 modes) and −8.6%-8.6\% (10 modes). Replacing it with standard min-max normalization into [−1,1][-1, 1] results in the largest drop (−25.7%-25.7\%).
    2. Conditional Sampling: Removing training-by-sampling (w/o S., sampling conditions according to raw training frequency rather than log-frequency) reduces performance by −17.8%-17.8\%, and leads to a complete failure (0%0\% F1) on the highly imbalanced credit dataset. Removing the condition vector entirely (w/o C.) drops classification performance by −36.5%-36.5\%.
    3. Network Architecture: Replacing WGAN-GP with standard GAN loss reduces performance by −6.5%-6.5\%. PacGAN stabilizes training for vanilla GANs (improving over GAN alone by 1.3%1.3\%) but has minor effect when combined with WGAN-GP (+1.75%+1.75\% for WGANGP only).

Coverage note — None. All primary contributions—including mode-specific normalization, the conditional generator and training-by-sampling scheme, the TVAE model, the evaluation framework, the full training algorithm, benchmark experimental comparisons, and ablation analyses—are included.

References

  1. 1.Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, 2017.
  2. 2.Laura Aviñó, Matteo Ruffini, and Ricard Gavaldà. Generating synthetic but plausible healthcare record datasets. In KDD workshop on Machine Learning for Medicine and Healthcare, 2018.
  3. 3.Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.
  4. 4.Ramiro Camino, Christian Hammerschmidt, and Radu State. Generating multi-categorical samples with generative adversarial networks. In ICML workshop on Theoretical Foundations and Applications of Deep Generative Models, 2018.
  5. 5.Zhengping Che, Yu Cheng, Shuangfei Zhai, Zhaonan Sun, and Yan Liu. Boosting deep learning risk prediction with generative adversarial networks for electronic health records. In International Conference on Data Mining. IEEE, 2017.
  6. 6.Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F. Stewart, and Jimeng Sun. Generating multi-label discrete patient records using generative adversarial networks. In Machine Learning for Healthcare Conference. PMLR, 2017.
  7. 7.C Chow and Cong Liu. Approximating discrete probability distributions with dependence trees. IEEE transactions on Information Theory, 14(3):462–467, 1968.
  8. 8.Graham Cormode, Cecilia Procopiuc, Divesh Srivastava, Entong Shen, and Ting Yu. Differentially private spatial decompositions. In International Conference on Data Engineering. IEEE, 2012.
  9. 9.Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
  10. 10.Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014.
  11. 11.Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems, 2017.
  12. 12.Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on International Conference on Machine Learning, 2015.
  13. 13.Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations, 2016.
  14. 14.James Jordon, Jinsung Yoon, and Mihaela van der Schaar. Pate-gan: Generating synthetic data with differential privacy guarantees. In International Conference on Learning Representations, 2019.
  15. 15.Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2013.
  16. 16.Yann LeCun and Corinna Cortes. MNIST handwritten digit database, 2010. URL http://yann.lecun.com/exdb/mnist/.
  17. 17.Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. Pacgan: The power of two samples in generative adversarial networks. In Advances in Neural Information Processing Systems, 2018.
  18. 18.Noseong Park, Mahmoud Mohammadi, Kshitij Gorde, Sushil Jajodia, Hongkyu Park, and Youngmin Kim. Data synthesis based on generative adversarial networks. In International Conference on Very Large Data Bases, 2018.
  19. 19.Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. The synthetic data vault. In International Conference on Data Science and Advanced Analytics. IEEE, 2016.
  20. 20.Jerome P Reiter. Using cart to generate partially synthetic public use microdata. Journal of Official Statistics, 21(3):441, 2005.
  21. 21.Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational learning. In Advances in Neural Information Processing Systems, 2017.
  22. 22.Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014.
  23. 23.Yi Sun, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Learning vine copula models for synthetic data generation. In AAAI Conference on Artificial Intelligence, 2018.
  24. 24.Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. In International Conference on Learning Representations, 2016.
  25. 25.Alexandre Yahi, Rami Vanguri, Noémie Elhadad, and Nicholas P Tatonetti. Generative adversarial networks for electronic health records: A framework for exploring and evaluating methods for predicting drug-induced laboratory test trajectories. In NIPS workshop on machine learning for health care, 2017.
  26. 26.Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In AAAI Conference on Artificial Intelligence, 2017.
  27. 27.Jun Zhang, Xiaokui Xiao, and Xing Xie. Privtree: A differentially private algorithm for hierarchical decompositions. In International Conference on Management of Data. ACM, 2016.
  28. 28.Jun Zhang, Graham Cormode, Cecilia M Procopiuc, Divesh Srivastava, and Xiaokui Xiao. Privbayes: Private data release via bayesian networks. ACM Transactions on Database Systems, 42(4):25, 2017.
  29. 29.Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In international conference on computer vision, pages 2223–2232. IEEE, 2017.

Citation

MLA
Xu, L., et al. “Modeling Tabular Data Using Conditional GAN”. arXiv, 2019, http://arxiv.org/abs/1907.00503v2.
APA
Xu, L., Skoularidou, M., Cuesta-Infante, A., & Veeramachaneni, K. (2019). Modeling Tabular data using Conditional GAN. arXiv. http://arxiv.org/abs/1907.00503v2
Chicago
Xu, L., M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni. 2019. “Modeling Tabular Data Using Conditional GAN”. arXiv. http://arxiv.org/abs/1907.00503v2.
Harvard
Xu, L. et al. (2019) “Modeling Tabular data using Conditional GAN”, arXiv [Preprint]. Available at: http://arxiv.org/abs/1907.00503v2.
Vancouver
1. Xu L, Skoularidou M, Cuesta-Infante A, Veeramachaneni K (2019) Modeling Tabular data using Conditional GAN. arXiv

BibTeX

@article{xu2019modeling,
  title = {Modeling Tabular data using Conditional GAN},
  author = {Xu, Lei and Skoularidou, Maria and Cuesta-Infante, Alfredo and Veeramachaneni, Kalyan},
  year = {2019},
  journal = {arXiv},
  url = {http://arxiv.org/abs/1907.00503v2},
  eprint = {1907.00503}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors