keyword
audio generation
Audio generation is the computational process of synthesizing acoustic signals, such as human speech, music, and environmental sound effects, using machine learning and artificial intelligence models. These systems learn the underlying temporal dependencies, spectral features, and statistical patterns of sound data to produce novel, coherent audio in the form of raw waveforms or time-frequency representations like spectrograms. The process can operate unconditionally or be guided by conditioning signals such as text prompts, speaker characteristics, musical notes, or duration parameters. Modern audio generation relies on various generative modeling architectures, including autoregressive networks, diffusion models, and variational autoencoders, enabling applications across speech synthesis, automated music composition, and virtual sound design.
2 items
Fast Timing-Conditioned Latent Audio Diffusion
Zach Evans, CJ Carr, Josiah Taylor, Scott H. Hawley, Jordi Pons
Why you should read this
Presents Stable Audio, a latent diffusion architecture conditioned on text and timing embeddings to generate variable-length, high-fidelity 44.1kHz stereo music and sound effects of up to 95 seconds in just 8 seconds of inference time.
Generating long-form 44.1kHz stereo audio from text prompts can be computationally demanding. Further, most previous works do not tackle that music and sound effects naturally vary in their duration. Our research focuses on the efficient generation of long-form, variable-length stereo music and sounds at 44.1kHz using text prompts with a generative model. Stable Audio is based on latent diffusion, with its latent defined by a fully-convolutional variational autoencoder. It is conditioned on text prompts as well as timing embeddings, allowing for fine control over the content and length of the generated music and sounds. Stable Audio is capable of rendering stereo signals of up to 95 sec at 44.1kHz in 8 sec on an A100 GPU. Despite its compute efficiency and fast inference, it is one of the best in two public text-to-music and -audio benchmarks and, differently from state-of-the-art models, can generate music with structure and stereo sounds.
Added
2026-09-28
WaveNet: A Generative Model for Raw Audio
Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alexander Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu
Why you should read this
Demonstrates that autoregressive principles extend perfectly to high-frequency continuous temporal data like raw audio through dilated causal convolutions.
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.
Added
2026-02-21