SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

Lantao YuWeinan ZhangJun WangYong Yu

article2016AAAI2,603 citations

Introduces SeqGAN to overcome the non-differentiability barrier of discrete sequence generation in generative adversarial networks by framing the generator as a reinforcement learning policy trained through policy gradients and Monte Carlo search.

Listen

The article addresses the challenge of generating realistic sequences of discrete tokens, such as text, poems, speeches, or music, which standard generative adversarial networks struggle with due to difficulties in propagating gradients through discrete outputs and evaluating partially completed sequences. This matters because sequence generation underpins applications like natural language creation and creative content production, yet common training methods like maximum likelihood estimation suffer from exposure bias and fail to capture long-term sequence quality effectively.

The article set out to develop and evaluate a framework called SeqGAN that adapts adversarial training to discrete sequence tasks. It treats the generator as a reinforcement learning policy that receives rewards from a discriminator assessing complete sequences, using Monte Carlo search to estimate values for intermediate steps.

Experiments involved pre-training on synthetic data generated by an oracle LSTM model and real-world datasets including Chinese poems, Obama speeches, and folk music tunes. Performance was measured against baselines such as maximum likelihood estimation, scheduled sampling, and policy gradient with BLEU scoring, using metrics like negative log-likelihood against the oracle, BLEU scores, mean squared error, and human expert judgments.

SeqGAN produced sequences with significantly lower negative log-likelihood scores than baselines on synthetic data and achieved higher BLEU scores plus better human ratings on real tasks, often approaching the quality of actual human-created poems. The adversarial signal proved more effective than fixed metrics like BLEU for guiding generation. These gains indicate that SeqGAN can create more natural and coherent sequences, reducing reliance on task-specific scoring functions and improving outcomes in creative or data-scarce domains.

The findings suggest organizations working on content generation tools should consider integrating policy-gradient methods with adversarial discriminators to boost output quality. Next steps include scaling the approach with Monte Carlo tree search and value networks for longer sequences and larger datasets. Training stability requires careful tuning of generator and discriminator update frequencies, and results on synthetic data may not fully generalize without further validation on diverse real-world distributions.

Shanghai Jiao Tong University University College London gan generative adversarial networks generative model Sequence Generative Adversarial Nets policy gradient adversarial training

arXiv: 1609.05473 LantaoYu/SeqGAN

Paper: Generative Adversarial Networks, Ian J. Goodfellow et al. (2014). It introduces the fundamental Generative Adversarial Network (GAN) minimax formulation and training dynamics that SeqGAN adapts for discrete token sequences.
Paper: Generating Sequences With Recurrent Neural Networks, Alex Graves (2013). It establishes recurrent neural networks and LSTM architectures for autoregressive sequential data generation, which serve as the generator and oracle baselines in SeqGAN.
Paper: Improved Techniques for Training GANs, Tim Salimans et al. (2016). It details stabilization heuristics and evaluation protocols for GANs that provide essential context for overcoming adversarial training collapse.
Paper: A Critical Review of Recurrent Neural Networks for Sequence Learning, Zachary C. Lipton et al. (2015). It synthesizes the architectural mechanics, gating systems, and sequence-modeling challenges of RNNs underlying SeqGAN's policy generator.
Paper: Generating Sentences from a Continuous Space, Samuel R. Bowman et al. (2016). It explores continuous latent-space modeling for sentences, highlighting the limitations of maximum likelihood estimation that SeqGAN resolves via reinforcement learning.

Paper: Improved Training of Wasserstein GANs, Ishaan Gulrajani et al. (2017). It introduces gradient-penalized Wasserstein GAN objectives, which substantially stabilize the adversarial feedback signals required in discrete and continuous generative domains.
Paper: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium, Martin Heusel et al. (2017). It presents the two time-scale update rule and the Fréchet Inception Distance, advancing the theoretical convergence guarantees and evaluation of adversarial generators.
Paper: Towards Principled Methods for Training Generative Adversarial Networks, Martin Arjovsky et al. (2017). It provides a rigorous mathematical analysis of discriminator vanishing gradients and training instability in GANs, offering formal foundations for why standard adversarial losses struggle.
Paper: Generative Adversarial Networks: An Overview, Antonia Creswell et al. (2017). It provides a comprehensive survey and taxonomy of adversarial training frameworks and structural extensions across diverse generative applications.
Paper: Structured Denoising Diffusion Models in Discrete State-Spaces, Jacob Austin et al. (2021). It develops structured diffusion models in discrete state-spaces as an alternative non-autoregressive paradigm to policy-gradient adversarial sequence generation.
Paper: Diffusion-LM Improves Controllable Text Generation, Xiang Lisa Li et al. (2022). It investigates controllable discrete text generation using continuous latent diffusion, addressing the discrete sequence problem without reinforcement learning rollouts.

Cover for SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

Abstract

As a new way of training generative models, Generative Adversarial Nets (GAN) that uses a discriminative model to guide the training of the generative model has enjoyed considerable success in generating real-valued data. However, it has limitations when the goal is for generating sequences of discrete tokens. A major reason lies in that the discrete outputs from the generative model make it difficult to pass the gradient update from the discriminative model to the generative model. Also, the discriminative model can only assess a complete sequence, while for a partially generated sequence, it is non-trivial to balance its current score and the future one once the entire sequence has been generated. In this paper, we propose a sequence generation framework, called SeqGAN, to solve the problems. Modeling the data generator as a stochastic policy in reinforcement learning (RL), SeqGAN bypasses the generator differentiation problem by directly performing gradient policy update. The RL reward signal comes from the GAN discriminator judged on a complete sequence, and is passed back to the intermediate state-action steps using Monte Carlo search. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over strong baselines.

Topics

gan generative adversarial networks generative model reinforcement learning stochastic synthetic data

Keywords

Sequence Generative Adversarial Nets policy gradient adversarial training sequence generation problem reinforcement learning recurrent neural networks natural language generation text generation music generation speech generation

Introduction
Related Work
Sequence Generative Adversarial Nets
SeqGAN via Policy Gradient
The Generative Model for Sequences
The Discriminative Model for Sequences
Synthetic Data Experiments
Evaluation Metric
Training Setting
Results
Discussion
Real-world Scenarios
Text Generation
Music Generation
Conclusion
Acknowledgments
References
A Appendix
Proof for Eq. (6)
Model Implementations
The Generative Model for Sequences
The Discriminative Model for Sequences
More Ablation Study

Knowls

Knowl 1 — SeqGAN Reinforcement Learning Framework for Sequence Generation
model/method
SeqGAN addresses the non-differentiability of discrete token generation in Generative Adversarial Networks (GANs) by casting sequence generation as a reinforcement learning (RL) sequential decision-making process.

Given a discrete vocabulary $\mathcal{Y}$ and a target sequence length $T$ , the generative model $G_\theta(y_t | Y_{1:t-1})$ parameterized by $\theta$ acts as an RL policy. At time step $t$ , the state $s$ consists of the prefix of generated tokens $Y_{1:t-1} = (y_1, \dots, y_{t-1})$ , and the action $a$ is the next token $y_t \in \mathcal{Y}$ . The state transition is deterministic: taking action $y_t$ in state $Y_{1:t-1}$ transitions to state $Y_{1:t} = (y_1, \dots, y_t)$ with probability $1$ .

A discriminative model $D_\phi(Y_{1:T})$ , parameterized by $\phi$ , outputs the probability that a complete sequence $Y_{1:T} = (y_1, \dots, y_T)$ is drawn from real data rather than generated. The discriminator serves as the reward function at the end of each episode (sequence generation), and policy gradient methods update $G_\theta$ to maximize the expected reward received from $D_\phi$ .
Knowl 2 — Action-Value Estimation via Monte Carlo Rollouts
equation
Because the discriminator $D_\phi$ can only evaluate a complete sequence $Y_{1:T} = (y_1, \dots, y_T)$ , the action-value function $Q_{D_\phi}^{G_\theta}(s, a)$ for an intermediate state $s = Y_{1:t-1}$ and action $a = y_t$ is estimated using Monte Carlo (MC) search with a roll-out policy $G_\beta$ :

$Q_{D_\phi}^{G_\theta}(s = Y_{1:t-1}, a = y_t) = \begin{cases} \frac{1}{N} \sum_{n=1}^N D_\phi(Y_{1:T}^n), & Y_{1:T}^n \in \mathrm{MC}^{G_\beta}(Y_{1:t}; N) \quad \text{for } t < T \\ D_\phi(Y_{1:T}) & \text{for } t = T \end{cases}$

where $\mathrm{MC}^{G_\beta}(Y_{1:t}; N) = \{Y_{1:T}^1, \dots, Y_{1:T}^N\}$ denotes an $N$ -time Monte Carlo rollout starting from the prefix $Y_{1:t} = (y_1, \dots, y_t)$ , where the remaining $T - t$ tokens $Y_{t+1:T}^n = (y_{t+1}^n, \dots, y_T^n)$ in each sample are generated by the roll-out policy $G_\beta$ . In standard practice, $G_\beta$ is configured to be identical to the current generator $G_\theta$ .
Knowl 3 — Policy Gradient for Generator Optimization in SeqGAN
equation
The objective of the generator $G_\theta$ starting from the initial state $s_0$ is to maximize the expected end reward:

$J(\theta) = \mathbb{E}[R_T | s_0, \theta] = \sum_{y_1 \in \mathcal{Y}} G_\theta(y_1 | s_0) \cdot Q_{D_\phi}^{G_\theta}(s_0, y_1)$

where $R_T$ is the sequence reward provided by the discriminator $D_\phi$ , and $\mathcal{Y}$ is the token vocabulary. Due to deterministic state transitions and zero intermediate rewards, the policy gradient with respect to the generator parameters $\theta$ is given by:

$\nabla_\theta J(\theta) = \mathbb{E}_{Y_{1:t-1} \sim G_\theta} \left[ \sum_{y_t \in \mathcal{Y}} \nabla_\theta G_\theta(y_t | Y_{1:t-1}) \cdot Q_{D_\phi}^{G_\theta}(Y_{1:t-1}, y_t) \right]$

Using the REINFORCE likelihood-ratio trick, an unbiased batch estimation of the gradient over an episode of length $T$ is computed as:

$\nabla_\theta J(\theta) \approx \frac{1}{T} \sum_{t=1}^T \mathbb{E}_{y_t \sim G_\theta(y_t | Y_{1:t-1})} \left[ \nabla_\theta \log G_\theta(y_t | Y_{1:t-1}) \cdot Q_{D_\phi}^{G_\theta}(Y_{1:t-1}, y_t) \right]$

The parameters are updated using gradient ascent $\theta \leftarrow \theta + \alpha_h \nabla_\theta J(\theta)$ with learning rate $\alpha_h \in \mathbb{R}^+$ .
Knowl 4 — SeqGAN Training Algorithm
algorithm
The complete training pipeline of SeqGAN consists of supervised pre-training followed by adversarial reinforcement learning updates:
Input: Training dataset S = {X_{1:T}}, generator G_\theta, rollout policy G_\beta, discriminator D_\phi
Output: Trained generator G_\theta
Initialize G_\theta and D_\phi with random weights \theta and \phi
Pre-train G_\theta using Maximum Likelihood Estimation (MLE) on S
\beta <- \theta
Generate negative samples using G_\theta for training D_\phi
Pre-train D_\phi by minimizing cross-entropy on S (positive) and generated samples (negative)
repeat
    for g_step = 1 to g-steps do
        Generate a sequence Y_{1:T} = (y_1, ..., y_T) ~ G_\theta
        for t = 1 to T do
            Compute Q(a = y_t, s = Y_{1:t-1}) via Monte Carlo rollouts with G_\beta and D_\phi
        end for
        Update generator parameters \theta via policy gradient: \theta <- \theta + \alpha \nabla_\theta J(\theta)
    end for
    for d_step = 1 to d-steps do
        Generate negative examples using current G_\theta and combine with positive examples S
        Train discriminator D_\phi for k epochs by minimizing cross-entropy
    end for
    \beta <- \theta
until SeqGAN converges
Knowl 5 — CNN Discriminator Architecture for Sequence Discrimination
model/method
SeqGAN employs a Convolutional Neural Network (CNN) as the sequence discriminator $D_\phi$ . Given an input sequence of tokens $x_1, \dots, x_T$ , the sequence is represented by concatenating $k$ -dimensional token embeddings:

$\mathcal{E}_{1:T} = x_1 \oplus x_2 \oplus \dots \oplus x_T \in \mathbb{R}^{T \times k}$

A convolutional kernel $w \in \mathbb{R}^{l \times k}$ with window size $l$ is applied over sub-matrices $\mathcal{E}_{i:i+l-1}$ to generate a feature map:

$c_i = \rho(w \otimes \mathcal{E}_{i:i+l-1} + b)$

where $\otimes$ denotes the Frobenius inner product (summation of element-wise products), $b$ is a bias scalar, and $\rho$ is a non-linear activation function. Max-over-time pooling produces a pooled feature $\tilde{c} = \max\{c_1, \dots, c_{T-l+1}\}$ . Multiple kernels of varying window sizes $l \in \{1, \dots, T\}$ are used.

A highway network layer is applied to the pooled feature vector, followed by a fully connected output layer with a sigmoid activation function to output the probability $D_\phi(Y_{1:T}) \in [0, 1]$ that the sequence is real. The discriminator parameters $\phi$ are trained to minimize binary cross-entropy:

$\min_\phi -\mathbb{E}_{Y \sim p_{\text{data}}}[\log D_\phi(Y)] - \mathbb{E}_{Y \sim G_\theta}[\log(1 - D_\phi(Y))]$
Knowl 6 — Oracle Evaluation Framework for Synthetic Sequence Generation
experimental setup
To evaluate generative sequence models under exact ground truth conditions, an oracle framework is used. A randomly initialized LSTM network with weights sampled from $\mathcal{N}(0, 1)$ acts as the oracle distribution $G_{\text{oracle}}(x_t | x_1, \dots, x_{t-1})$ . The oracle generates $10,000$ sequences of length $T = 20$ to form the training set $S$ for the generative models.

The generative quality is evaluated using the negative log-likelihood under the oracle distribution ( $NLL_{\text{oracle}}$ ) on $100,000$ sequences sampled from the trained generator $G_\theta$ :

$NLL_{\text{oracle}} = -\mathbb{E}_{Y_{1:T} \sim G_\theta} \left[ \sum_{t=1}^T \log G_{\text{oracle}}(y_t | Y_{1:t-1}) \right]$

A lower $NLL_{\text{oracle}}$ indicates that the generated sequences match the true data distribution $G_{\text{oracle}}$ more closely.

Knowl 7 — Synthetic Sequence Generation Benchmark Results

data/table

The performance of sequence generative models was evaluated on synthetic data generated by an LSTM oracle, measured by oracle negative log-likelihood ( $NLL_{\text{oracle}}$ ) on 100,000 test sequences. Statistical significance was verified with paired t-tests comparing SeqGAN to each baseline.

SeqGAN outperforms standard Maximum Likelihood Estimation (MLE), Scheduled Sampling (SS), and Policy Gradient with BLEU reward (PG-BLEU) with statistical significance ( $p < 10^{-6}$ ). This demonstrates that the dynamically learned adversarial discriminator reward guides sequence generation more effectively than either maximum likelihood objectives or fixed heuristic reward metrics like BLEU.

Knowl 8 — Real-World Text Generation Performance on Chinese Poetry and Political Speeches

data/table

SeqGAN was evaluated on two text generation tasks: Chinese quatrain composition (16,394 poems, 4 lines of 5 characters, 20 characters total) and Obama political speech generation (11,092 paragraphs). Chinese poems were evaluated via BLEU-2 and a blind human evaluation where 70 literary experts assigned binary scores (+1 for real, 0 for machine) across mixed batches of 20 real poems, 20 MLE poems, and 20 SeqGAN poems. Obama speech texts were evaluated via BLEU-3 and BLEU-4 against the test corpus.

Chinese Poem Generation	Human score	$p$ -value	BLEU-2	$p$ -value
MLE	0.4165	0.0034	0.6670	$< 10^{-6}$
SeqGAN	0.5356	–	0.7389	–
Real data	0.6011	–	0.7460	–
Obama Speech Generation	BLEU-3	$p$ -value	BLEU-4	$p$ -value
MLE	0.519	$< 10^{-6}$	0.416	0.00014
SeqGAN	0.556	–	0.427	–

SeqGAN significantly outperforms MLE across all automated BLEU metrics and human expert judgment, with SeqGAN's human evaluation score (0.5356) approaching that of real human-authored poems (0.6011).

Knowl 9 — Real-World Music Generation Performance

data/table

SeqGAN was evaluated on symbolic music generation using the Nottingham dataset (695 folk tune solo tracks). Pitch sequences were represented by 88 discrete values corresponding to piano keys, sampled every 0.4 seconds to form sequences of length 32. Performance was evaluated using BLEU-4 (measuring discrete pitch pattern similarity) and Mean Squared Error (MSE, measuring continuous pitch curve fitness).

SeqGAN achieves a significantly higher BLEU-4 score and a lower MSE compared to MLE training, indicating superior modeling of both local token patterns and global musical pitch trajectories.

Knowl 10 — Training Dynamics and Discriminator Bootstrapping in SeqGAN
empirical result
The stability and convergence of SeqGAN depend heavily on the balance between generator updates ( $g$ -steps), discriminator update iterations ( $d$ -steps), and discriminator epochs per step ( $k$ ):
1. When $g$ -steps are much larger than $d$ -steps and $k$ (e.g., $g=100, d=1, k=10$ ), the generator updates too rapidly for the discriminator to remain optimal, leading to misleading gradient rewards and unstable training.
2. Updating the discriminator more frequently with higher epoch counts (e.g., $g=30, d=1, k=30$ , or $g=1, d=1, k=10$ ) stabilizes training.
3. Utilizing multiple negative sample batches per discriminator step ( $d > 1$ , e.g., $g=1, d=5, k=3$ ), where fresh negative sequences from $G_\theta$ are paired with the fixed positive training set $S$ in a bootstrapping fashion, provides the most stable convergence and lowest oracle negative log-likelihood. This empirically confirms the GAN theoretical requirement that the discriminator must stay near its optimum given $G_\theta$ to provide meaningful training signals.

Coverage note — None was omitted; all key contributions including framework formulation, mathematical objectives, rollout value estimation, algorithm, network architectures, and synthetic and real-world experiments (poetry, speeches, music) are fully covered.

References

1.Bachman, P., and Precup, D. 2015. Data generation as sequential decision making. In NIPS, 3249–3257.
2.Bahdanau, D.; Brakel, P.; Xu, K.; et al. 2016. An actor-critic algorithm for sequence prediction. arXiv:1607.07086.
3.Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473.
4.Bengio, Y.; Yao, L.; Alain, G.; and Vincent, P. 2013. Generalized denoising auto-encoders as generative models. In NIPS, 899–907.
5.Bengio, S.; Vinyals, O.; Jaitly, N.; and Shazeer, N. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In NIPS, 1171–1179.
6.Browne, C. B.; Powley, E.; Whitehouse, D.; Lucas, S. M.; et al. 2012. A survey of monte carlo tree search methods. IEEE TCIAIG 4(1):1–43.
7.Cho, K.; Van Merriënboer, B.; Gulcehre, C.; et al. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. EMNLP.
8.Denton, E. L.; Chintala, S.; Fergus, R.; et al. 2015. Deep generative image models using a laplacian pyramid of adversarial networks. In NIPS, 1486–1494.
9.Glynn, P. W. 1990. Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM 33(10):75–84.
10.Goodfellow, I., et al. 2014. Generative adversarial nets. In NIPS, 2672–2680.
11.Goodfellow, I.; Bengio, Y.; and Courville, A. 2016. Deep learning. 2015.
12.Goodfellow, I. 2016. Generative adversarial networks for text. http://goo.gl/Wg9DR7.
13.Graves, A. 2013. Generating sequences with recurrent neural networks. arXiv:1308.0850.
14.He, J.; Zhou, M.; and Jiang, L. 2012. Generating chinese classical poems with statistical machine translation models. In AAAI.
15.Hingston, P. 2009. A turing test for computer game bots. IEEE TCIAIG 1(3):169–186.
16.Hinton, G. E.; Osindero, S.; and Teh, Y.-W. 2006. A fast learning algorithm for deep belief nets. Neural computation 18(7):1527–1554.
17.Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural computation 9(8):1735–1780.
18.Huszár, F. 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? arXiv:1511.05101.
19.Kim, Y. 2014. Convolutional neural networks for sentence classification. arXiv:1408.5882.
20.Kingma, D. P., and Welling, M. 2014. Auto-encoding variational bayes. ICLR.
21.Lai, S.; Xu, L.; Liu, K.; and Zhao, J. 2015. Recurrent convolutional neural networks for text classification. In AAAI, 2267–2273.
22.Manaris, B.; Roos, P.; Machado, P.; et al. 2007. A corpus-based hybrid approach to music analysis and composition. In NCAI, volume 22, 839.
23.Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, 311–318.
24.Quinlan, J. R. 1996. Bagging, boosting, and c4. 5. In AAAI/IAAI, Vol. 1, 725–730.
25.Salakhutdinov, R. 2009. Learning deep generative models. Ph.D. Dissertation, University of Toronto.
26.Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; et al. 2016. Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489.
27.Srivastava, N.; Hinton, G. E.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014. Dropout: a simple way to prevent neural networks from overfitting. JMLR 15(1):1929–1958.
28.Srivastava, R. K.; Greff, K.; and Schmidhuber, J. 2015. Highway networks. arXiv:1505.00387.
29.Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NIPS, 3104–3112.
30.Sutton, R. S.; McAllester, D. A.; Singh, S. P.; Mansour, Y.; et al. 1999. Policy gradient methods for reinforcement learning with function approximation. In NIPS, 1057–1063.
31.Veselỳ, K.; Ghoshal, A.; Burget, L.; and Povey, D. 2013. Sequence-discriminative training of deep neural networks. In INTERSPEECH, 2345–2349.
32.Wen, T.-H.; Gasic, M.; Mrksic, N.; Su, P.-H.; Vandyke, D.; and Young, S. 2015. Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. arXiv:1508.01745.
33.Williams, R. J. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8(3-4):229–256.
34.Yi, X.; Li, R.; and Sun, M. 2016. Generating chinese classical poems with RNN encoder-decoder. arXiv:1604.01537.
35.Zhang, X., and Lapata, M. 2014. Chinese poetry generation with recurrent neural networks. In EMNLP, 670–680.
36.Zhang, X., and LeCun, Y. 2015. Text understanding from scratch. arXiv:1502.01710.

Citation

MLA

Yu, L., et al. “SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient”. Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017, https://doi.org/10.1609/aaai.v31i1.10804.

APA

Yu, L., Zhang, W., Wang, J., & Yu, Y. (2017). SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10804

Chicago

Yu, L., W. Zhang, J. Wang, and Y. Yu. 2017. “SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient”. Proceedings of the AAAI Conference on Artificial Intelligence 31 (1). https://doi.org/10.1609/aaai.v31i1.10804.

Harvard

Yu, L. et al. (2017) “SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient”, Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). Available at: https://doi.org/10.1609/aaai.v31i1.10804.

Vancouver

1. Yu L, Zhang W, Wang J, Yu Y (2017) SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence. https://doi.org/10.1609/aaai.v31i1.10804

BibTeX

@article{Yu_2017, title={SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient}, volume={31}, ISSN={2159-5399}, url={http://dx.doi.org/10.1609/aaai.v31i1.10804}, DOI={10.1609/aaai.v31i1.10804}, number={1}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, publisher={Association for the Advancement of Artificial Intelligence (AAAI)}, author={Yu, Lantao and Zhang, Weinan and Wang, Jun and Yu, Yong}, year={2017}, month=Feb }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by-sa/4.0/

Algorithm	Random	MLE	SS	PG-BLEU	SeqGAN
NLL	10.310	9.038	8.985	8.946	8.736
$p$ -value	$< 10^{-6}$	$< 10^{-6}$	$< 10^{-6}$	$< 10^{-6}$	–

SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — SeqGAN Reinforcement Learning Framework for Sequence Generation

Knowl 2 — Action-Value Estimation via Monte Carlo Rollouts

Knowl 3 — Policy Gradient for Generator Optimization in SeqGAN

Knowl 4 — SeqGAN Training Algorithm

Knowl 5 — CNN Discriminator Architecture for Sequence Discrimination

Knowl 6 — Oracle Evaluation Framework for Synthetic Sequence Generation

Knowl 7 — Synthetic Sequence Generation Benchmark Results

Knowl 8 — Real-World Text Generation Performance on Chinese Poetry and Political Speeches

Knowl 9 — Real-World Music Generation Performance

Knowl 10 — Training Dynamics and Discriminator Bootstrapping in SeqGAN

References

Citation

BibTeX

Source Code

Access the Paper