HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Jungil KongJaehyeon KimJaekyoung Bae

article2020NeurIPS2,833 citations

Introduces HiFi-GAN, a GAN-based neural vocoder that models periodic audio patterns to match the synthesis quality of autoregressive baselines while generating high-fidelity speech over one hundred times faster than real time.

Listen

The article addresses the challenge of creating natural-sounding speech from text using neural networks, where current methods either produce high-quality audio too slowly, require excessive computing resources, or fall short in realism. This matters because voice interfaces are expanding rapidly in assistants, vehicles, and smart devices, demanding both lifelike output and fast, efficient generation.

The work set out to develop a generative adversarial network called HiFi-GAN that delivers both higher audio quality and greater speed than leading autoregressive and flow-based models for converting mel-spectrograms into raw waveforms. The approach involved training a fully convolutional generator with multi-receptive field fusion modules alongside two discriminators—one that examines periodic patterns at multiple intervals and another that assesses audio at different scales—using adversarial, mel-spectrogram, and feature-matching losses. Experiments used the LJSpeech single-speaker dataset for primary evaluation, the VCTK multi-speaker set for generalization tests, crowd-sourced mean opinion score ratings, and direct speed measurements on GPU and CPU hardware.

Key findings show that HiFi-GAN variants consistently outperformed publicly available WaveNet, WaveGlow, and MelGAN implementations in perceptual quality while running substantially faster. The largest model reached a mean opinion score of 4.36 against a human ground-truth score of 4.45 and generated audio 167.9 times faster than real time on a V100 GPU. A compact version used only 0.92 million parameters yet maintained strong quality, and the smallest model produced audio 13.4 times faster than real time on CPU with quality comparable to an autoregressive baseline. The models also generalized effectively to unseen speakers and improved further when fine-tuned for end-to-end pipelines.

These results indicate that explicit modeling of periodic signal patterns enables GANs to close the quality gap with slower methods without sacrificing efficiency, supporting lower-latency and lower-memory applications such as on-device synthesis. The article notes that the same discriminator setup works across generator configurations of varying sizes, allowing flexible trade-offs between speed and quality.

The authors recommend releasing the code as open source to support further research and suggest that practitioners select generator variants according to target hardware constraints. Main limitations include reliance on specific English-language datasets and the need for fine-tuning in fully end-to-end systems; confidence in the core quality and speed claims is high given consistent human ratings and controlled comparisons, though real-world deployment may require additional validation on diverse accents and conditions.

Kakao Enterprise audio cpu gan High Fidelity Speech Synthesis Generative Adversarial Networks neural speech synthesis

arXiv: 2010.05646 jik876/hifi-gan

Paper: WaveNet: A Generative Model for Raw Audio, Aäron van den Oord et al. (2016). WaveNet established the foundational neural audio synthesis benchmark that HiFi-GAN explicitly sets out to outperform in speed and efficiency.
Paper: Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions, Jonathan Shen et al. (2017). Tacotron 2 established mel-spectrogram conditioning as the standard intermediate representation for neural vocoding pipelines that HiFi-GAN adopts.
Paper: Generative Adversarial Networks, Ian J. Goodfellow et al. (2014). This seminal work introduces the foundational generative adversarial network framework and minimax training objective utilized by HiFi-GAN.
Paper: Tacotron: Towards End-to-End Speech Synthesis, Yuxuan Wang et al. (2017). Tacotron introduced the modern deep learning paradigm for neural text-to-speech synthesis pipelines in which neural vocoders like HiFi-GAN operate.

Paper: DiffWave: A Versatile Diffusion Model for Audio Synthesis, Zhifeng Kong et al. (2021). DiffWave explores diffusion models as an alternative non-autoregressive paradigm to GAN vocoders like HiFi-GAN for fast, high-fidelity waveform generation.
Paper: Qwen3-TTS Technical Report, Hangrui Hu et al. (2026). Qwen3-TTS extends high-fidelity speech synthesis to massive multi-speaker, multilingual streaming architectures in modern large language model frameworks.

Cover for HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Abstract

Several recent work on speech synthesis have employed generative adversarial networks (GANs) to produce raw waveforms. Although such methods improve the sampling efficiency and memory usage, their sample quality has not yet reached that of autoregressive and flow-based generative models. In this work, we propose HiFi-GAN, which achieves both efficient and high-fidelity speech synthesis. As speech audio consists of sinusoidal signals with various periods, we demonstrate that modeling periodic patterns of an audio is crucial for enhancing sample quality. A subjective human evaluation (mean opinion score, MOS) of a single speaker dataset indicates that our proposed method demonstrates similarity to human quality while generating 22.05 kHz high-fidelity audio 167.9 times faster than real-time on a single V100 GPU. We further show the generality of HiFi-GAN to the mel-spectrogram inversion of unseen speakers and end-to-end speech synthesis. Finally, a small footprint version of HiFi-GAN generates samples 13.4 times faster than real-time on CPU with comparable quality to an autoregressive counterpart.

Topics

audio cpu gan generative adversarial networks generative model human evaluation mean opinion scores sinusoidal signals speech signals audio stream

Keywords

High Fidelity Speech Synthesis Generative Adversarial Networks neural speech synthesis mel-spectrogram domain raw waveform MoL WaveNet

1 Introduction
2 HiFi-GAN
2.1 Overview
2.2 Generator
2.3 Discriminator
2.4 Training Loss Terms
3 Experiments
4 Results
4.1 Audio Quality and Synthesis Speed
4.2 Ablation Study
4.3 Generalization to Unseen Speakers
4.4 End-to-End Speech Synthesis
5 Conclusion
References

Knowls

Knowl 1 — Multi-Period Discriminator Architecture
model/method
The Multi-Period Discriminator (MPD) is an ensemble of sub-discriminators designed to identify diverse periodic patterns in raw audio waveforms. Because speech audio comprises superpositions of sinusoidal signals with various periodicities, MPD employs multiple sub-discriminators where each individual sub-discriminator processes only periodic samples spaced by a specific period $p$ .

To minimize overlap between the sub-discriminators' receptive fields, prime periods are selected: $p \in \{2, 3, 5, 7, 11\}$ . Given a 1D raw waveform $x$ of length $T$ , the sub-discriminator with period $p$ first reshapes $x$ into a 2D representation of height $T/p$ and width $p$ . Then, 2D convolutional layers are applied with kernel size restricted to $k \times 1$ along the temporal (height) dimension, preserving independence across the periodic (width) dimension.

Each sub-discriminator consists of a stack of strided 2D convolutional layers with Leaky ReLU activations and Weight Normalization applied to all layers. Reshaping the 1D input into a 2D tensor instead of sub-sampling allows gradients from MPD to propagate back to every time step in the input audio.
Knowl 2 — Multi-Receptive Field Fusion Module
model/method
The Multi-Receptive Field Fusion (MRF) module is a core building block in the HiFi-GAN generator that observes audio patterns across multiple temporal lengths in parallel.

In the generator network, every transposed convolution upsampling layer is followed by an MRF module. The MRF module consists of $|k_r|$ parallel residual blocks (ResBlocks), each configured with a distinct 1D convolution kernel size $k_r[n]$ and a specific set of dilation rates $D_r[n]$ . The output of the MRF module is the sum of the outputs of these parallel residual blocks:

$\text{MRF}(x) = \sum_{n=1}^{|k_r|} \text{ResBlock}_n(x)$

Within each residual block, 1D dilated convolutions with Leaky ReLU activations are stacked with residual connections. By combining different kernel sizes and dilation patterns concurrently, the generator processes diverse receptive field sizes simultaneously.
Knowl 3 — HiFi-GAN Training Objectives and Loss Functions
equation
HiFi-GAN is trained adversarially using a combination of Least-Squares GAN (LS-GAN) loss, Feature Matching loss, and Mel-Spectrogram reconstruction loss across all sub-discriminators of the Multi-Period Discriminator (MPD) and Multi-Scale Discriminator (MSD).

Let $x$ denote the ground-truth raw waveform, $s$ the conditioning mel-spectrogram, $G$ the generator, and $D_k$ the $k$ -th sub-discriminator out of $K$ total sub-discriminators across MPD and MSD. The adversarial objectives for $D_k$ and $G$ are defined as:

$\mathcal{L}_{\text{Adv}}(D_k; G) = \mathbb{E}_{(x, s)} \left[ (D_k(x) - 1)^2 + (D_k(G(s)))^2 \right]$

$\mathcal{L}_{\text{Adv}}(G; D_k) = \mathbb{E}_{s} \left[ (D_k(G(s)) - 1)^2 \right]$

The feature matching loss measures the L1 distance between the intermediate feature representations of $D_k$ for real and generated waveforms across $T_k$ layers:

$\mathcal{L}_{\text{FM}}(G; D_k) = \mathbb{E}_{(x, s)} \left[ \sum_{i=1}^{T_k} \frac{1}{N_{k,i}} \| D_k^i(x) - D_k^i(G(s)) \|_1 \right]$

where $D_k^i$ and $N_{k,i}$ denote the extracted feature map and the number of features in the $i$ -th layer of $D_k$ , respectively.

The mel-spectrogram loss is the L1 distance between the mel-spectrogram of the generated audio and that of the ground-truth audio:

$\mathcal{L}_{\text{Mel}}(G) = \mathbb{E}_{(x, s)} \left[ \| \phi(x) - \phi(G(s)) \|_1 \right]$

where $\phi$ is the deterministic transformation from a raw waveform to its corresponding mel-spectrogram.

The total generator loss $\mathcal{L}_G$ and discriminator loss $\mathcal{L}_D$ are:

$\mathcal{L}_G = \sum_{k=1}^K \left[ \mathcal{L}_{\text{Adv}}(G; D_k) + \lambda_{\text{fm}} \mathcal{L}_{\text{FM}}(G; D_k) \right] + \lambda_{\text{mel}} \mathcal{L}_{\text{Mel}}(G)$

$\mathcal{L}_D = \sum_{k=1}^K \mathcal{L}_{\text{Adv}}(D_k; G)$

where the loss weighting hyperparameters are set to $\lambda_{\text{fm}} = 2$ and $\lambda_{\text{mel}} = 45$ .

Knowl 4 — HiFi-GAN Generator Architecture Configurations

model/method

The HiFi-GAN generator is a fully convolutional neural network that accepts an 80-band mel-spectrogram as input and upsamples it through a sequence of transposed 1D convolutions followed by Multi-Receptive Field Fusion (MRF) modules. The architecture provides three variants ( $V1$ , $V2$ , and $V3$ ) balancing fidelity and computational speed.

Model	$h_u$	$k_u$	$k_r$	$D_r$
$V1$	512	$[16, 16, 4, 4]$	$[3, 7, 11]$	$[[1, 1], [3, 1], [5, 1]] \times 3$
$V2$	128	$[16, 16, 4, 4]$	$[3, 7, 11]$	$[[1, 1], [3, 1], [5, 1]] \times 3$
$V3$	256	$[16, 16, 8]$	$[3, 5, 7]$	$[[1], [2]], [[2], [6]], [[3], [12]]$

In the table, $h_u$ denotes the initial hidden channel dimension of transposed convolutions, $k_u$ is the list of transposed convolution kernel sizes (which also dictate the temporal upsampling strides), $k_r$ is the list of kernel sizes for the residual blocks in each MRF module, and $D_r$ is the list of dilation rates in the residual blocks.

In $V1$ and $V2$ , each residual block stacks 2 convolutional layers with 1 residual connection, repeated 3 times. In $V3$ , each residual block stacks 1 convolutional layer with 1 residual connection, repeated 2 times, resulting in a substantially shallower network.

Knowl 5 — Multi-Scale Discriminator Architecture
model/method
The Multi-Scale Discriminator (MSD) evaluates continuous audio signals across different temporal resolutions to capture consecutive audio patterns and long-term dependencies.

MSD consists of three identical sub-discriminators operating on three scales of input audio:
1. Raw audio waveform ( $1\times$ scale).
2. $2\times$ downsampled audio using average pooling.
3. $4\times$ downsampled audio using average pooling.
Each sub-discriminator is constructed from a stack of strided and grouped 1D convolutional layers with Leaky ReLU activations. To stabilize training, Spectral Normalization is applied to the first sub-discriminator (operating on raw audio), whereas Weight Normalization is applied to the second and third sub-discriminators (operating on downsampled audio). While MPD inspects disjoint periodic slices of the waveform, MSD evaluates smoothed, continuous sequences.

Knowl 6 — Single-Speaker Audio Synthesis Quality and Inference Speed Comparison

data/table

HiFi-GAN variants ( $V1$ , $V2$ , and $V3$ ) were evaluated on the single-speaker LJSpeech dataset (16-bit PCM, 22.05 kHz) and compared against autoregressive (WaveNet with Mixture of Logistics), flow-based (WaveGlow), and GAN-based (MelGAN) vocoders using Mean Opinion Score (MOS, with 95% confidence intervals) and synthesis speed on CPU (Intel i7 2.6 GHz) and GPU (single NVIDIA V100).

Model	MOS (CI)	Speed on CPU (kHz)	Speed on GPU (kHz)	# Param (M)
Ground Truth	4.45 ( $\pm 0.06$ )	–	–	–
WaveNet (MoL)	4.02 ( $\pm 0.08$ )	–	0.07 ( $\times 0.003$ )	24.73
WaveGlow	3.81 ( $\pm 0.08$ )	4.72 ( $\times 0.21$ )	501 ( $\times 22.75$ )	87.73
MelGAN	3.79 ( $\pm 0.09$ )	145.52 ( $\times 6.59$ )	14,238 ( $\times 645.73$ )	4.26
HiFi-GAN $V1$	4.36 ( $\pm 0.07$ )	31.74 ( $\times 1.43$ )	3,701 ( $\times 167.86$ )	13.92
HiFi-GAN $V2$	4.23 ( $\pm 0.07$ )	214.97 ( $\times 9.74$ )	16,863 ( $\times 764.80$ )	0.92
HiFi-GAN $V3$	4.05 ( $\pm 0.08$ )	296.38 ( $\times 13.44$ )	26,169 ( $\times 1,186.80$ )	1.46

Speed is expressed in kHz ( $1000$ raw samples/sec) and relative to real-time ( $imes$ ). HiFi-GAN $V1$ achieves a MOS of 4.36, closely approaching human ground truth (4.45). $V2$ delivers 4.23 MOS with only 0.92M parameters. $V3$ runs 13.44 times faster than real-time on CPU and 1,186.8 times faster on GPU while outperforming WaveNet.

Knowl 7 — Ablation Study of HiFi-GAN Architectural Components

data/table

An ablation study using the HiFi-GAN $V3$ generator configuration trained up to 500k steps on LJSpeech demonstrates the individual contribution of the Multi-Period Discriminator (MPD), Multi-Scale Discriminator (MSD), Multi-Receptive Field Fusion (MRF), mel-spectrogram loss, and prime period selection.

Model	MOS (CI)
Ground Truth	4.57 ( $\pm 0.04$ )
Baseline (HiFi-GAN $V3$ )	4.10 ( $\pm 0.05$ )
w/o MPD	2.28 ( $\pm 0.09$ )
w/o MSD	3.74 ( $\pm 0.05$ )
w/o MRF	3.92 ( $\pm 0.05$ )
w/o Mel-Spectrogram Loss	3.25 ( $\pm 0.05$ )
MPD $p=[2,4,8,16,32]$	3.90 ( $\pm 0.05$ )
MelGAN	2.88 ( $\pm 0.08$ )
MelGAN with MPD	3.35 ( $\pm 0.07$ )

Removing MPD causes the largest drop in perceptual quality ( $4.10 \to 2.28$ ), confirming that modeling periodicity is critical. Removing mel-spectrogram loss also substantially degrades performance ( $4.10 \to 3.25$ ). Choosing prime periods ( $p=[2,3,5,7,11]$ ) outperforms power-of-two periods ( $p=[2,4,8,16,32]$ ) by 0.20 MOS. Adding MPD to MelGAN improves MelGAN's MOS from 2.88 to 3.35.

Knowl 8 — Zero-Shot Mel-Spectrogram Inversion on Unseen Multi-Speaker Speech

data/table

To assess generalization to unseen speakers, models were trained on the VCTK dataset downsampled to 22 kHz with 9 speakers excluded from training and subsequently evaluated on 50 utterances from those 9 held-out speakers.

Model	MOS (CI)
Ground Truth	3.79 ( $\pm 0.07$ )
WaveNet (MoL)	3.52 ( $\pm 0.08$ )
WaveGlow	3.52 ( $\pm 0.08$ )
MelGAN	3.50 ( $\pm 0.08$ )
HiFi-GAN $V1$	3.77 ( $\pm 0.07$ )
HiFi-GAN $V2$	3.69 ( $\pm 0.07$ )
HiFi-GAN $V3$	3.61 ( $\pm 0.07$ )

All three HiFi-GAN variants ( $V1$ : 3.77, $V2$ : 3.69, $V3$ : 3.61) outperform WaveNet, WaveGlow, and MelGAN on unseen speakers, with $V1$ scoring within 0.02 of the ground truth.

Knowl 9 — End-to-End Speech Synthesis and Domain Adaptation Fine-Tuning

data/table

When HiFi-GAN vocoders are paired with Tacotron2 in a two-stage text-to-speech pipeline, acoustic mismatch between ground-truth mel-spectrograms and Tacotron2-generated mel-spectrograms occurs. Fine-tuning the vocoder on Tacotron2-predicted mel-spectrograms in teacher-forcing mode for 100k steps resolves this degradation.

Model	MOS (CI)
Ground Truth	4.23 ( $\pm 0.07$ )
WaveGlow (w/o fine-tuning)	3.69 ( $\pm 0.08$ )
HiFi-GAN $V1$ (w/o fine-tuning)	3.91 ( $\pm 0.08$ )
HiFi-GAN $V2$ (w/o fine-tuning)	3.88 ( $\pm 0.08$ )
HiFi-GAN $V3$ (w/o fine-tuning)	3.89 ( $\pm 0.08$ )
WaveGlow (fine-tuned)	3.66 ( $\pm 0.08$ )
HiFi-GAN $V1$ (fine-tuned)	4.18 ( $\pm 0.08$ )
HiFi-GAN $V2$ (fine-tuned)	4.12 ( $\pm 0.07$ )
HiFi-GAN $V3$ (fine-tuned)	4.02 ( $\pm 0.08$ )

Without fine-tuning, HiFi-GAN variants achieve 3.88--3.91 MOS. After fine-tuning on predicted mel-spectrograms, HiFi-GAN $V1$ reaches 4.18 MOS (near ground-truth 4.23), while WaveGlow shows no improvement (3.66 MOS).

Knowl 10 — Frequency Response Mechanism of MPD vs MSD
empirical result
In a toy sinc reconstruction experiment and synthetic sinusoidal classification tasks, the Multi-Period Discriminator (MPD) avoids the high-frequency spectral loss inherent to the Multi-Scale Discriminator (MSD).

MSD downsamples input waveforms via average pooling, which functions as a low-pass filter that dampens high frequencies. As a consequence, sub-discriminators operating at $2\times$ and $4\times$ scales receive smoothed signals where high-frequency generation errors are attenuated, making poor synthetic waveforms appear indistinguishable from ground truth in those branches.

In contrast, MPD reshapes 1D signals into 2D representations without smoothing, preserving the spectral magnitude of high frequencies (subject only to aliasing). In a controlled binary classification experiment classifying 40,000 sinusoidal signals where 99.9% were true frequencies and 0.1% were error frequencies, MPD identified the 0.1% false frequencies with 85.33% accuracy on unseen validation data, whereas MSD achieved only 50.42% accuracy.

Coverage note — None was omitted; all key architectural components, training losses, model variants, benchmark tables, ablation findings, and analytical experiments from the paper are represented.

References

1.Mikołaj Bińkowski, Jeff Donahue, Sander Dieleman, Aidan Clark, Erich Elsen, Norman Casagrande, Luis C Cobo, and Karen Simonyan. High fidelity speech synthesis with adversarial networks. arXiv preprint arXiv:1909.11646, 2019.
2.Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial audio synthesis. arXiv preprint arXiv:1802.04208, 2018.
3.Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
4.Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017.
5.Keith Ito. The lj speech dataset. https://keithito.com/LJ-Speech-Dataset/, 2017.
6.Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, pages 10215–10224, 2018.
7.Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brébisson, Yoshua Bengio, and Aaron C Courville. Melgan: Generative adversarial networks for conditional waveform synthesis. In Advances in Neural Information Processing Systems 32, pages 14910–14921, 2019.
8.Rithesh Kumar. descriptinc/melgan-neurips. https://github.com/descriptinc/melgan-neurips, 2019.
9.Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using a learned similarity metric. In International conference on machine learning, pages 1558–1566. PMLR, 2016.
10.Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, and Ming Liu. Neural speech synthesis with transformer network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6706–6713, 2019.
11.Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
12.Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 2794–2802, 2017.
13.Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440, 2015.
14.Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957, 2018.
15.Aaron Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George Driessche, Edward Lockhart, Luis Cobo, Florian Stimberg, et al. Parallel wavenet: Fast high-fidelity speech synthesis. In International conference on machine learning, pages 3918–3926. PMLR, 2018.
16.Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, 2016.
17.Wei Ping, Kainan Peng, Andrew Gibiansky, Sercan O Arik, Ajay Kannan, Sharan Narang, Jonathan Raiman, and John Miller. Deep voice 3: Scaling text-to-speech with convolutional sequence learning. arXiv preprint arXiv:1710.07654, 2017.
18.Wei Ping, Kainan Peng, and Jitong Chen. Clarinet: Parallel wave generation in end-to-end text-to-speech. arXiv preprint arXiv:1807.07281, 2018.
19.Ryan Prenger, Rafael Valle, and Bryan Catanzaro. Waveglow: A flow-based generative network for speech synthesis. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3617–3621. IEEE, 2019.
20.Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in neural information processing systems, pages 901–909, 2016.
21.Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al. Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4779–4783. IEEE, 2018.
22.Mingxing Tan, Ruoming Pang, and Quoc V Le. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10781–10790, 2020.
23.Rafael Valle. Nvidia/tacotron2. https://github.com/NVIDIA/tacotron2, 2018a.
24.Rafael Valle. Nvidia/waveglow. https://github.com/NVIDIA/waveglow, 2018b.
25.Christophe Veaux, Junichi Yamagishi, Kirsten MacDonald, et al. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit. University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2017.
26.Ryuichi Yamamoto. wavenet vocoder. https://github.com/r9y9/wavenet_vocoder/, 2018.
27.Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6199–6203. IEEE, 2020.
28.Bohan Zhai, Tianren Gao, Flora Xue, Daniel Rothchild, Bichen Wu, Joseph E Gonzalez, and Kurt Keutzer. Squeezewave: Extremely lightweight vocoders for on-device speech synthesis. arXiv preprint arXiv:2001.05685, 2020.

Citation

MLA

Kong, J., et al. “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis”. Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 17022–33, https://proceedings.neurips.cc/paper_files/paper/2020/file/c5d736809766d46260d816d8dbc9eb44-Paper.pdf.

APA

Kong, J., Kim, J., & Bae, J. (2020). HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. Advances in Neural Information Processing Systems, 33, 17022–17033. https://proceedings.neurips.cc/paper_files/paper/2020/file/c5d736809766d46260d816d8dbc9eb44-Paper.pdf

Chicago

Kong, J., J. Kim, and J. Bae. 2020. “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis”. Advances in Neural Information Processing Systems 33: 17022–33. https://proceedings.neurips.cc/paper_files/paper/2020/file/c5d736809766d46260d816d8dbc9eb44-Paper.pdf.

Harvard

Kong, J., Kim, J. and Bae, J. (2020) “HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 17022–17033. Available at: https://proceedings.neurips.cc/paper_files/paper/2020/file/c5d736809766d46260d816d8dbc9eb44-Paper.pdf.

Vancouver

1. Kong J, Kim J, Bae J (2020) HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 17022–17033

BibTeX

@inproceedings{kong2020hifi,
  title = {HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis},
  author = {Kong, Jungil and Kim, Jaehyeon and Bae, Jaekyoung},
  year = {2020},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {33},
  pages = {17022-17033},
  url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/c5d736809766d46260d816d8dbc9eb44-Paper.pdf}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Multi-Period Discriminator Architecture

Knowl 2 — Multi-Receptive Field Fusion Module

Knowl 3 — HiFi-GAN Training Objectives and Loss Functions

Knowl 4 — HiFi-GAN Generator Architecture Configurations

Knowl 5 — Multi-Scale Discriminator Architecture

Knowl 6 — Single-Speaker Audio Synthesis Quality and Inference Speed Comparison

Knowl 7 — Ablation Study of HiFi-GAN Architectural Components

Knowl 8 — Zero-Shot Mel-Spectrogram Inversion on Unseen Multi-Speaker Speech

Knowl 9 — End-to-End Speech Synthesis and Domain Adaptation Fine-Tuning

Knowl 10 — Frequency Response Mechanism of MPD vs MSD

References

Citation

BibTeX

Source Code

Access the Paper