Generative Adversarial Networks: An Overview

Antonia CreswellTom WhiteVincent DumoulinKai ArulkumaranBiswa SenguptaAnil A Bharath

article2017IEEE Signal Processing Magazine3,863 citations

Surveys the theoretical foundations, architectural variations, and training strategies of generative adversarial networks, connecting key applications like image synthesis and super-resolution to open research challenges through a signal processing perspective.

Listen

Generative adversarial networks offer a method for learning rich data representations from largely unlabelled examples by pitting two neural networks against each other. One network, the generator, produces synthetic samples intended to match the distribution of real data; the other, the discriminator, learns to distinguish real samples from the synthetic ones. The approach matters because many practical tasks in image analysis, editing and classification lack the large annotated datasets that supervised deep learning normally requires.

This review paper sets out to introduce the core ideas, architectures and training issues of GANs to readers in signal processing, while drawing explicit links to familiar tools such as principal-component analysis and wavelet representations. The authors synthesise the literature published between the original 2014 formulation and late 2017, covering fully connected and convolutional designs, conditional variants, inference models such as ALI and BiGAN, and alternative cost functions including Wasserstein and f-divergence formulations. They illustrate the methods with published image-synthesis, classification and translation results rather than new experiments of their own.

The principal findings are that suitably architected GANs can produce visually convincing images at modest computational cost, that latent-space vectors learned by the generator often support semantically meaningful operations such as attribute editing, and that several practical heuristics—feature matching, instance noise, one-sided label smoothing and batch normalisation—materially improve training stability. At the same time, the review shows that mode collapse, vanishing gradients and the absence of reliable quantitative evaluation metrics remain widespread. Architectures that incorporate an explicit encoder or that replace the original Jensen-Shannon objective with a Wasserstein distance appear more robust than the earliest designs.

These results imply that large collections of unlabelled imagery can now be turned into reusable feature representations and into tools for image manipulation without hand-crafted losses for every new task. For organisations that hold extensive image archives but limited labelling budgets, the technology therefore lowers the cost and time needed to build vision systems. At present, however, successful deployment still demands careful hyper-parameter tuning and domain-specific validation, because no single training recipe yet guarantees convergence or diversity across data sets.

Further progress requires both theoretical work on convergence guarantees and saddle-point optimisation and the development of agreed evaluation protocols that allow reliable comparison of models. In the near term, practitioners should test at least the Wasserstein and adversarial-autoencoder variants on representative data before committing to production use, and should budget for continued monitoring of sample diversity and downstream task performance. The review itself is a synthesis of published methods rather than a source of new measurements, so readers should treat its claims about relative ease of training as informed expert judgement rather than statistically validated benchmarks.

Cortexica Vision Systems Ltd.Imperial College London Mila Université de Montréal Victoria University of Wellington backpropagation algorithm classification generative adversarial networks Generative Adversarial Network Generator Discriminator

arXiv: 1710.07035

Paper: Generative Adversarial Networks, Ian J. Goodfellow et al. (2014). Reading the original Generative Adversarial Networks paper provides the essential foundational definition of the adversarial framework and minimax game reviewed in the source.
Paper: Conditional Generative Adversarial Nets, Mehdi Mirza et al. (2014). Understanding conditional generative adversarial nets is a prerequisite for grasping the conditional variants and applications discussed in the overview.
Paper: Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, Alec Radford et al. (2016). Familiarity with deep convolutional GAN architectures provides the core technical background required to follow the image modeling approaches surveyed in the source.
Paper: Wasserstein Generative Adversarial Networks, Martin Arjovsky et al. (2017). Reviewing Wasserstein GANs offers necessary context on alternative objective functions and stability improvements that the source paper categorizes and evaluates.
Paper: InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, Xi Chen et al. (2016). Understanding InfoGAN's information-maximizing approach to unsupervised representation learning clarifies the feature learning capabilities highlighted in the overview.
Paper: Improved Techniques for Training GANs, Tim Salimans et al. (2016). Knowing the heuristic stabilization techniques developed for training GANs helps contextualize the training challenges and solutions reviewed in the source.

Paper: Progressive Growing of GANs for Improved Quality, Stability, and Variation, Tero Karras et al. (2018). Progressively growing GANs directly extends the architectural and stability principles outlined in the source to achieve unprecedented high-resolution image synthesis.
Paper: A Style-Based Generator Architecture for Generative Adversarial Networks, Tero Karras et al. (2019). The style-based generator architecture builds upon the foundational GAN formulations reviewed in the source to enable fine-grained control over synthesized image features.
Paper: Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, Jun-Yan Zhu et al. (2017). Cycle-consistent adversarial networks build directly upon the conditional and mapping concepts discussed in the source to solve unpaired image-to-image translation.
Paper: Self-Attention Generative Adversarial Networks, Han Zhang et al. (2019). Self-attention GANs extend the convolutional models covered in the overview by introducing global context mechanisms that improve image generation fidelity.
Paper: Large Scale GAN Training for High Fidelity Natural Image Synthesis, Andrew Brock et al. (2019). Large-scale GAN training scales up the baseline methods and challenges identified in the source to achieve state-of-the-art fidelity on diverse natural images.
Paper: Spectral Normalization for Generative Adversarial Networks, Takeru Miyato et al. (2018). Spectral normalization applies mathematical Lipschitz constraints to advance the stability of discriminator training beyond the methods surveyed in the overview.

Cover for Generative Adversarial Networks: An Overview

Abstract

Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this through deriving backpropagation signals through a competitive process involving a pair of networks. The representations that can be learned by GANs may be used in a variety of applications, including image synthesis, semantic image editing, style transfer, image super-resolution and classification. The aim of this review paper is to provide an overview of GANs for the signal processing community, drawing on familiar analogies and concepts where possible. In addition to identifying different methods for training and constructing GANs, we also point to remaining challenges in their theory and application.

Topics

backpropagation algorithm classification generative adversarial networks image super-resolution signal processing semantic images network architecture

Keywords

Generative Adversarial Network Generator Discriminator Latent Space Training Mode Collapse Wasserstein GAN Conditional GAN Adversarial Autoencoder Image Synthesis

I. INTRODUCTION
II. PRELIMINARIES
A. Terminology
B. Notation
C. Capturing Data Distributions
*D. Related Work*
III. GAN ARCHITECTURES
*A. Fully Connected GANs*
*B. Convolutional GANs*
C. Conditional GANs
D. GANs with Inference Models
E. Adversarial Autoencoders (AAE)
IV. TRAINING GANs
A. Introduction
B. Training Tricks
C. Alternative formulations
D. A Brief Comparison of GAN Variants
V. THE STRUCTURE OF LATENT SPACE
VI. APPLICATIONS OF GANs
A. Classification and Regression
B. Image Synthesis
C. Image-to-image translation
D. Super-resolution
VII. DISCUSSION
A. Open Questions

Knowls

Knowl 1 — Minimax Objective Formulation of Generative Adversarial Networks
model/method
A Generative Adversarial Network (GAN) consists of two competing differentiable functions: a generator $G : \mathbb{R}^{|z|} \to \mathbb{R}^{|x|}$ parameterized by weights $\Theta_G$ , and a discriminator $D : \mathbb{R}^{|x|} \to (0, 1)$ parameterized by weights $\Theta_D$ . The generator maps a latent noise vector $z \sim p_z(z)$ to synthetic data samples $x = G(z)$ with an induced probability distribution $p_g(x)$ , while the discriminator estimates the probability that an observation $x \in \mathbb{R}^{|x|}$ was drawn from the true data distribution $p_{\text{data}}(x)$ rather than $p_g(x)$ .

The two networks are optimized simultaneously according to a two-player minimax game with value function $V(G, D)$ : $\min_G \max_D V(G, D) = \mathbb{E}_{x \sim p_{\text{data}}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 - D(G(z)))]$

To prevent vanishing gradients during early training phases when the discriminator easily rejects generated samples, the minimax objective for updating $G$ is commonly replaced with the non-saturating objective $\max_G \mathbb{E}_{z \sim p_z(z)}[\log D(G(z))]$ .
Knowl 2 — Optimal Discriminator and Equivalence to Jensen-Shannon Divergence Minimization
theoretical result
For any fixed generator $G$ with induced data distribution $p_g(x)$ , the optimal discriminator $D^*(x)$ that maximizes the value function $V(G, D) = \mathbb{E}_{x \sim p_{\text{data}}(x)}[\log D(x)] + \mathbb{E}_{x \sim p_g(x)}[\log(1 - D(x))]$ is uniquely determined by: $D^*(x) = \frac{p_{\text{data}}(x)}{p_{\text{data}}(x) + p_g(x)}$

When the discriminator is optimal ( $D = D^*$ ), the generator's optimization problem reduces to minimizing the Jensen-Shannon divergence $D_{\text{JS}}$ between the true data distribution $p_{\text{data}}$ and the generated distribution $p_g$ : $V(G, D^*) = -\log(4) + 2 \cdot D_{\text{JS}}(p_{\text{data}} \parallel p_g)$ where $D_{\text{JS}}(p_{\text{data}} \parallel p_g) = \frac{1}{2} D_{\text{KL}}\left(p_{\text{data}} \,\parallel\, \frac{p_{\text{data}} + p_g}{2}\right) + \frac{1}{2} D_{\text{KL}}\left(p_g \,\parallel\, \frac{p_{\text{data}} + p_g}{2}\right)$ . The global minimum occurs if and only if $p_g(x) = p_{\text{data}}(x)$ , where $V(G, D^*) = -\log(4)$ and $D^*(x) = 0.5$ for all inputs $x$ in the support of $p_{\text{data}}$ .
Knowl 3 — Training Instability Caused by Low-Dimensional Support Mismatch
theoretical result
In high-dimensional representation spaces $\mathbb{X} \subset \mathbb{R}^{|x|}$ (such as image spaces $\mathbb{R}^{3N^2}$ for $N \times N$ colour images), the supports of the empirical data distribution $p_{\text{data}}(x)$ and the generator distribution $p_g(x)$ are confined to lower-dimensional manifolds. Consequently, the supports of $p_{\text{data}}(x)$ and $p_g(x)$ have measure zero in $\mathbb{X}$ and almost always exhibit empty intersection or disjoint supports.

When the supports of $p_{\text{data}}$ and $p_g$ do not overlap, there exists a discriminator $D$ that can separate real samples $x \sim p_{\text{data}}(x)$ from fake samples $x \sim p_g(x)$ with $100\%$ accuracy. In this regime, the discriminator loss converges to zero, and the discriminator gradients with respect to the inputs vanish. Because the generator parameters $\Theta_G$ can only receive gradient feedback transmitted through the discriminator, the update gradients $\nabla_{\Theta_G} V(G, D)$ vanish, leaving the generator with no supervisory signal.
Knowl 4 — Wasserstein GAN and 1-Lipschitz Critic Optimization
model/method
The Wasserstein Generative Adversarial Network (WGAN) replaces the Jensen-Shannon divergence with the Earth Mover's (Wasserstein-1) distance $W(p_{\text{data}}, p_g) = \inf_{\gamma \in \Pi(p_{\text{data}}, p_g)} \mathbb{E}_{(x, y) \sim \gamma}[\|x - y\|]$ . Via the Kantorovich-Rubinstein duality, the training objective is formulated as: $\min_G \max_{D \in \mathcal{D}_L} \mathbb{E}_{x \sim p_{\text{data}}(x)}[D(x)] - \mathbb{E}_{z \sim p_z(z)}[D(G(z))]$ where $\mathcal{D}_L$ denotes the family of 1-Lipschitz continuous functions, and $D$ acts as a scalar critic rather than a binary classifier.

To satisfy the 1-Lipschitz condition on the critic $D$ parameterized by $\Theta_D$ , two primary mechanisms are used:
1. Weight Clipping: Clamping network weights $\Theta_D$ to a compact interval $[-c, c]$ after each gradient step, which limits the function class of the critic.
2. Gradient Penalty (WGAN-GP): Adding a regularization penalty to the critic objective enforcing unit gradient norms along lines between data and model distributions: $\mathcal{L}_{\text{GP}} = \mathbb{E}_{\hat{x} \sim p_{\hat{x}}}\left[\left(\|\nabla_{\hat{x}} D(\hat{x})\|_2 - 1\right)^2\right]$ where $\hat{x} = \epsilon x + (1 - \epsilon) G(z)$ for $\epsilon \sim \mathcal{U}(0, 1)$ , $x \sim p_{\text{data}}$ , and $z \sim p_z$ . This eliminates vanishing gradients across non-overlapping distributions.
Knowl 5 — Adversarially Learned Inference and Bidirectional GANs
model/method
Standard GANs do not provide an explicit inference mapping to invert data samples $x \in \mathbb{R}^{|x|}$ into latent representations $z \in \mathbb{R}^{|z|}$ . Adversarially Learned Inference (ALI) and Bidirectional GANs (BiGAN) introduce an inference network (encoder) $E : \mathbb{R}^{|x|} \to \mathbb{R}^{|z|}$ alongside the generative network (decoder) $G : \mathbb{R}^{|z|} \to \mathbb{R}^{|x|}$ .

The discriminator $D(x, z)$ operates over joint data-latent pairs rather than data samples alone. The adversarial game is defined over the joint distribution $q(x, z) = p_{\text{data}}(x) q_E(z \mid x)$ and the generative joint distribution $p(x, z) = p_z(z) p_G(x \mid z)$ : $\min_{G, E} \max_D \mathbb{E}_{x \sim p_{\text{data}}, \hat{z} \sim q_E(z \mid x)}[\log D(x, \hat{z})] + \mathbb{E}_{z \sim p_z, \hat{x} \sim p_G(x \mid z)}[\log(1 - D(\hat{x}, z))]$

At the theoretical optimum of this minimax formulation, the joint distributions match ( $q(x, z) = p(x, z)$ ), ensuring that the encoder $E$ learns to invert the generator $G$ ( $E = G^{-1}$ ) almost everywhere.
Knowl 6 — Variational f-Divergence Minimization in f-GANs
model/method
The $f$ -GAN framework generalizes GAN objectives to minimize arbitrary $f$ -divergences between the data distribution $P = p_{\text{data}}$ and model distribution $Q = p_g$ . For a convex, lower semi-continuous function $f : \mathbb{R}_+ \to \mathbb{R}$ with $f(1) = 0$ , the $f$ -divergence is $D_f(P \parallel Q) = \int q(x) f\left(\frac{p(x)}{q(x)}\right) dx$ .

Using the Fenchel conjugate $f^*(t) = \sup_{u \in \text{dom}(f)} \{ut - f(u)\}$ , a variational lower bound on $D_f(P \parallel Q)$ is formulated as: $D_f(P \parallel Q) \ge \sup_{T \in \mathcal{T}} \left( \mathbb{E}_{x \sim P}[T(x)] - \mathbb{E}_{x \sim Q}[f^*(T(x))] \right)$ where $\mathcal{T}$ is a neural discriminator parameterized as $T(x) = g_f(V(x))$ with activation $g_f : \mathbb{R} \to \text{dom}(f^*)$ .

The generator $G$ and discriminator $V$ solve the minimax problem: $\min_G \max_V \mathbb{E}_{x \sim p_{\text{data}}(x)}[g_f(V(x))] - \mathbb{E}_{z \sim p_z(z)}[f^*(g_f(V(G(z))))]$
Knowl 7 — Adversarial Autoencoders and Adversarial Variational Bayes
model/method
Adversarial Autoencoders (AAEs) and Adversarial Variational Bayes (AVB) use adversarial objectives to shape the latent representation space $z \in \mathbb{R}^{|z|}$ of an autoencoder, avoiding the analytical restrictions of the Kullback-Leibler (KL) divergence used in Variational Autoencoders (VAEs).

In an AAE, an encoder $E(x)$ produces aggregated posterior latent vectors $q(z) = \int q_E(z \mid x) p_{\text{data}}(x) dx$ , and a decoder $G(z)$ reconstructs the input by minimizing a reconstruction loss $\|x - G(E(x))\|^2$ . Concurrently, a discriminator $D_z$ is trained to distinguish between latent samples drawn from an arbitrary prior distribution $p(z)$ and encoded samples $z \sim q(z)$ , while $E$ is trained to fool $D_z$ .

Adversarial Variational Bayes (AVB) generalizes this by using an auxiliary discriminator $T(x, z)$ to estimate the intractable log-density ratio $\log \frac{q(z \mid x)}{p(z)}$ , which enables exact variational inference over arbitrary flexible posterior distributions without requiring closed-form density functions.
Knowl 8 — Heuristic Stabilization Techniques for GAN Training
model/method
Several training heuristics have been established to prevent discriminator saturation, balance generator-discriminator capacity, and reduce non-convergence:
1. Feature Matching: Modifies the generator objective to minimize the mean squared difference between expected activations at an intermediate layer $f(x)$ of the discriminator: $\min_G \left\| \mathbb{E}_{x \sim p_{\text{data}}}[f(x)] - \mathbb{E}_{z \sim p_z}[f(G(z))] \right\|_2^2$
2. Minibatch Discrimination: Augments the discriminator's inputs with cross-sample distance features within each batch, allowing the discriminator to detect if the generator outputs multiple nearly identical samples.
3. Historical Averaging: Penalizes each network's parameters $\Theta$ if they deviate from their historical moving average $\frac{1}{t} \sum_{i=1}^t \Theta^{(i)}$ , damping parameter oscillations.
4. One-Sided Label Smoothing: Replaces the binary real target 1.0 with a relaxed target (such as 0.9) while keeping the fake target at 0.0, which softens the discriminator classification boundary.
5. Instance Noise: Injects decaying Gaussian noise $\epsilon \sim \mathcal{N}(0, \sigma^2 I)$ into both real and fake inputs prior to discriminator evaluation, broadening the support manifolds of $p_{\text{data}}$ and $p_g$ to ensure non-zero gradient overlap.
Knowl 9 — Mode Collapse and Saddle-Point Instabilities in Alternating Optimization
limitation
GAN training routinely exhibits mode collapse, in which the generator maps diverse latent codes $z$ to a small subset of data modes (partial collapse) or to a single repeated output (complete collapse), ignoring other modes of $p_{\text{data}}(x)$ .

This behavior arises because GAN training searches for a saddle point of a two-player zero-sum game rather than a local minimum of a single objective. The Hessian of the joint minimax objective is indefinite. First-order stochastic gradient descent applied alternately to generator parameters $\Theta_G$ and discriminator parameters $\Theta_D$ fails to converge to saddle points with probability one from random initializations, and the Jacobian of the gradient vector field often exhibits eigenvalues with zero real parts and large imaginary components, causing oscillatory or divergent parameter trajectories.

Unrolled GANs mitigate this instability by updating the generator parameters with respect to the discriminator's unrolled state $k$ steps into the future, enabling the generator to account for the discriminator's anticipated counter-updates rather than cycling between individual modes.
Knowl 10 — Conditioning Mechanisms and Latent Space Vector Arithmetic
model/method
Latent representations learned by GANs capture continuous geometric manifolds where semantic attributes can be steered through conditioning and vector arithmetic:
1. Conditional GAN (cGAN): Feeds an auxiliary conditioning variable $c$ (such as a class label or spatial mask) directly into both the generator $G(z, c)$ and the discriminator $D(x, c)$ , allowing targeted generation across distinct classes.
2. InfoGAN: Disentangles latent representations in an unsupervised fashion by splitting the noise vector into an incompressible noise source $z$ and structured latent codes $c$ . The objective maximizes a variational lower bound on the mutual information $I(c; G(z, c))$ using an auxiliary network head $Q(c \mid x)$ sharing layers with the discriminator.
3. Latent Vector Arithmetic: Because semantic attributes map to directional vectors in latent space $\mathbb{R}^{|z|}$ , attribute modification is performed by calculating a concept vector $v = \bar{z}_{\text{positive}} - \bar{z}_{\text{negative}}$ (the difference between mean latent vectors of images with and without the attribute) and generating manipulated images via $x_{\text{new}} = G(z + \alpha v)$ for scaling parameter $\alpha$ .

Coverage note — Domain-specific applied vision architectures (such as GAWWN, LAPGAN multiscale pyramids, CycleGAN style transfer, and SRGAN single-image super-resolution) were excluded as standalone knowls because they represent application-specific combinations of conditional GANs and loss formulations rather than core architectural or theoretical foundations of the GAN framework.

References

1.I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014, pp. 2672–2680.
2.J.-Y. Zhu, P. Krähenbühl, E. Shechtman, and A. A. Efros, “Generative visual manipulation on the natural image manifold,” in European Conference on Computer Vision. Springer, 2016, pp. 597–613.
3.K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan, “Unsupervised pixel-level domain adaptation with generative adversarial networks,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016.
4.J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-toimage translation using cycle-consistent adversarial networks,” in Proceedings of the International Conference on Computer Vision, 2017. [Online]. Available: https://arxiv.org/abs/1703.10593
5.A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” in Proceedings of the 5th International Conference on Learning Representations (ICLR) - workshop track, 2016.
6.A. Creswell and A. A. Bharath, “Adversarial training for sketch retrieval,” in Computer Vision – ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part I. Springer International Publishing, 2016.
7.Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
8.H. Hotelling, “Analysis of a complex of statistical variables into principal components.” Journal of educational psychology, vol. 24, no. 6, p. 417, 1933.
9.I. J. Goodfellow, “On distinguishability criteria for estimating generative models,” International Conference on Learning Representations - workshop track, 2015.
10.M. Gutmann and A. Hyvärinen, “Noise-contrastive estimation: A new estimation principle for unnormalized statistical models.” in AISTATS, vol. 1, no. 2, 2010, p. 6.
11.Y. Bengio, L. Yao, G. Alain, and P. Vincent, “Generalized denoising auto-encoders as generative models,” in Advances in Neural Information Processing Systems, 2013, pp. 899–907.
12.I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,” 2016, presented at the Neural Information Processing Systems Conference. [Online]. Available: https://arxiv.org/abs/ 1701.00160
13.E. L. Denton, S. Chintala, R. Fergus et al., “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in Neural Information Processing Systems, 2015, pp. 1486–1494.
14.J. Wu, C. Zhang, T. Xue, B. Freeman, and J. Tenenbaum, “Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling,” in Advances in Neural Information Processing Systems, 2016, pp. 82–90.
15.M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014.
16.X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, “Infogan: Interpretable representation learning by information maximizing generative adversarial nets,” in Advances in Neural Information Processing Systems, 2016.
17.A. Creswell and A. A. Bharath, “Inverting the generator of a generative adversarial network,” in NIPS Workshop on Adversarial Training, 2016.
18.Z. C. Lipton and S. Tripathi, “Precise recovery of latent vectors from generative adversarial networks,” in ICLR (workshop track), 2017.
19.V. Dumoulin, I. Belghazi, B. Poole, O. Mastropietro, A. Lamb, M. Arjovsky, and A. Courville, “Adversarially learned inference,” in (accepted, to appear) Proceedings of the International Conference on Learning Representations, 2017.
20.J. Donahue, P. Krähenbühl, and T. Darrell, “Adversarial feature learning,” in (accepted, to appear) Proceedings of the International Conference on Learning Representations, 2017.
21.C. Li, H. Liu, C. Chen, Y. Pu, L. Chen, R. Henao, and L. Carin, “Towards understanding adversarial learning for joint distribution matching,” in Advances in Neural Information Processing Systems, 2017.
22.A. Makhzani, J. Shlens, N. Jaitly, and I. Goodfellow, “Adversarial autoencoders,” in International Conference on Learning Representations (to appear), 2016. [Online]. Available: http://arxiv.org/abs/1511.05644
23.D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in Proceedings of the 2nd International Conference on Learning Representations (ICLR), 2014.
24.L. M. Mescheder, S. Nowozin, and A. Geiger, “Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks,” 2017. [Online]. Available: http://arxiv.org/abs/1701.04722
25.T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” in Advances in Neural Information Processing Systems, 2016, pp. 2226–2234.
26.M. Arjovsky and L. Bottou, “Towards principled methods for training generative adversarial networks,” NIPS 2016 Workshop on Adversarial Training, 2016.
27.E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 4, pp. 640–651, 2017.
28.S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proceedings of The 32nd International Conference on Machine Learning, 2015, pp. 448–456.
29.C. K. Sønderby, J. Caballero, L. Theis, W. Shi, and F. Huszár, “Amortised map inference for image super-resolution,” in International Conference on Learning Representations, 2017.
30.S. Nowozin, B. Cseke, and R. Tomioka, “f-gan: Training generative neural samplers using variational divergence minimization,” in Advances in Neural Information Processing Systems, 2016, pp. 271–279.
31.M. Uehara, I. Sato, M. Suzuki, K. Nakayama, and Y. Matsuo, “Generative adversarial nets from a density ratio estimation perspective,” arXiv preprint arXiv:1610.02920, 2016.
32.M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein GAN,” in Proceedings of The 34nd International Conference on Machine Learning, 2017.
33.I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, “Improved training of wasserstein gans,” in (accepted, to appear) Advances in Neural Information Processing Systems, 2017.
34.T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” in International Conference on Learning Representations, 2013.
35.S. Gurumurthy, R. K. Sarvadevabhatla, and V. B. Radhakrishnan, “Deligan: Generative adversarial networks for diverse and limited data,” in IEEE Conference On Computer Vision and Pattern Recognition (CVPR), 2017.
36.C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” in IEEE Conference on Computer Vision and Pattern Recognition, 2017.
37.X. Yu and F. Porikli, “Ultra-resolving face images by discriminative generative networks,” in European Conference on Computer Vision. Springer, 2016, pp. 318–333.
38.——, “Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3760–3768.
39.A. Shrivastava, T. Pfister, O. Tuzel, J. Susskind, W. Wang, and R. Webb, “Learning from simulated and unsupervised images through adversarial training,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
40.M. Zhang, K. T. Ma, J. H. Lim, Q. Zhao, and J. Feng, “Deep future gaze: Gaze anticipation on egocentric videos using adversarial networks,” in IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4372–4381.
41.M.-Y. Liu and O. Tuzel, “Coupled generative adversarial networks,” in Advances in neural information processing systems, 2016, pp. 469–477.
42.X. Huang, Y. Li, O. Poursaeed, J. Hopcroft, and S. Belongie, “Stacked generative adversarial networks,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016.
43.S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, “Generative adversarial text to image synthesis,” in International Conference on Machine Learning, 2016. [Online]. Available: https://arxiv.org/abs/1605.05396
44.S. E. Reed, Z. Akata, S. Mohan, S. Tenka, B. Schiele, and H. Lee, “Learning what and where to draw,” in Advances in Neural Information Processing Systems, 2016, pp. 217–225.
45.A. Brock, T. Lim, J. M. Ritchie, and N. Weston, “Neural photo editing with introspective adversarial networks,” in Proceedings of the 6th International Conference on Learning Representations (ICLR), 2017.
46.P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
47.C. Li and M. Wand, “Precomputed real-time texture synthesis with Markovian generative adversarial networks,” in European Conference on Computer Vision. Springer, 2016, pp. 702–716.
48.S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang, “Generalization and equilibrium in generative adversarial nets (gans),” in Proceedings of The 34nd International Conference on Machine Learning, 2017.
49.I. Tolstikhin, S. Gelly, O. Bousquet, C.-J. Simon-Gabriel, and B. Schölkopf, “Adagan: Boosting generative models,” Tech. Rep., 2017.
50.J. Zhao, M. Mathieu, and Y. LeCun, “Energy-based generative adversarial network,” in International Conference on Learning Representations, 2017. [Online]. Available: https://arxiv.org/abs/ 1609.03126
51.L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein, “Unrolled generative adversarial networks,” in Proceedings of the International Conference on Learning Representations, 2017. [Online]. Available: https://arxiv.org/abs/1611.02163
52.J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht, “Gradient descent only converges to minimizers,” in Conference on Learning Theory, 2016, pp. 1246–1257.
53.R. Pemantle, “Nonconvergence to unstable points in urn models and stochastic approximations,” Ann. Probab., vol. 18, no. 2, pp. 698–712, 04 1990.
54.L. M. Mescheder, S. Nowozin, and A. Geiger, “The numerics of gans,” in Advances in Neural Information Processing Systems, 2017. [Online]. Available: http://arxiv.org/abs/1705.10461
55.L. Theis, A. van den Oord, and M. Bethge, “A note on the evaluation of generative models,” in Proceedings of the International Conference on Learning Representations.

Citation

MLA

Creswell, A., et al. “Generative Adversarial Networks: An Overview”. IEEE Signal Processing Magazine, vol. 35, no. 1, 2018, pp. 53–65, https://doi.org/10.1109/MSP.2017.2765202.

APA

Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine, 35(1), 53–65. https://doi.org/10.1109/MSP.2017.2765202

Chicago

Creswell, A., T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath. 2018. “Generative Adversarial Networks: An Overview”. IEEE Signal Processing Magazine 35 (1): 53–65. https://doi.org/10.1109/MSP.2017.2765202.

Harvard

Creswell, A. et al. (2018) “Generative Adversarial Networks: An Overview”, IEEE Signal Processing Magazine, 35(1), pp. 53–65. Available at: https://doi.org/10.1109/MSP.2017.2765202.

Vancouver

1. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine 35:53–65

BibTeX

@article{Creswell_2018, title={Generative Adversarial Networks: An Overview}, volume={35}, ISSN={1558-0792}, url={http://dx.doi.org/10.1109/MSP.2017.2765202}, DOI={10.1109/msp.2017.2765202}, number={1}, journal={IEEE Signal Processing Magazine}, publisher={Institute of Electrical and Electronics Engineers (IEEE)}, author={Creswell, Antonia and White, Tom and Dumoulin, Vincent and Arulkumaran, Kai and Sengupta, Biswa and Bharath, Anil A.}, year={2018}, month=Jan, pages={53–65} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Generative Adversarial Networks: An Overview

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Minimax Objective Formulation of Generative Adversarial Networks

Knowl 2 — Optimal Discriminator and Equivalence to Jensen-Shannon Divergence Minimization

Knowl 3 — Training Instability Caused by Low-Dimensional Support Mismatch

Knowl 4 — Wasserstein GAN and 1-Lipschitz Critic Optimization

Knowl 5 — Adversarially Learned Inference and Bidirectional GANs

Knowl 6 — Variational f-Divergence Minimization in f-GANs

Knowl 7 — Adversarial Autoencoders and Adversarial Variational Bayes

Knowl 8 — Heuristic Stabilization Techniques for GAN Training

Knowl 9 — Mode Collapse and Saddle-Point Instabilities in Alternating Optimization

Knowl 10 — Conditioning Mechanisms and Latent Space Vector Arithmetic

References

Citation

BibTeX

Access the Paper