keyword
acoustic models
An acoustic model is a computational component used in speech processing and automatic speech recognition systems that represents the statistical relationship between an audio speech signal and the basic linguistic units of spoken language. It analyzes raw audio waveforms or extracted spectral features, such as spectrograms and filter bank energies, and maps these sound inputs to corresponding phonemes, syllables, subwords, or characters. While traditional systems relied on statistical frameworks combining Gaussian mixture models with hidden Markov models, modern acoustic modeling primarily utilizes deep neural networks and self-supervised learning architectures to capture complex acoustic variations across different speakers, accents, and recording environments. These models can operate as an independent stage alongside pronunciation lexicons and language models in modular speech recognizers, or they can be integrated directly into end-to-end speech recognition pipelines.
2 items
Toward a realistic model of speech processing in the brain with self-supervised learning
Juliette Millet, Charlotte Caucheteux, Pierre Orhan, Yves Boubenec, Alexandre Gramfort, Ewan Dunbar, Christophe Pallier, Jean-Remi King
Why you should read this
Demonstrates that self-supervised neural networks trained on raw audio learn brain-like cortical hierarchies and functional specializations from realistic amounts of unlabeled speech, offering a biologically plausible computational framework for human language acquisition.
Several deep neural networks have recently been shown to generate activations similar to those of the brain in response to the same input. These algorithms, however, remain largely implausible: they require (1) extraordinarily large amounts of data, (2) unobtainable supervised labels, (3) textual rather than raw sensory input, and / or (4) implausibly large memory (e.g. thousands of contextual words). These elements highlight the need to identify algorithms that, under these limitations, would suffice to account for both behavioral and brain responses. Focusing on speech processing, we here hypothesize that self-supervised algorithms trained on the raw waveform constitute a promising candidate. Specifically, we compare a recent self-supervised model, wav2vec 2.0, to the brain activity of 412 English, French, and Mandarin individuals recorded with functional Magnetic Resonance Imaging (fMRI), while they listened to approximately one hour of audio books. First, we show that this algorithm learns brain-like representations with as little as 600 hours of unlabelled speech – a quantity comparable to what infants can be exposed to during language acquisition. Second, its functional hierarchy aligns with the cortical hierarchy of speech processing. Third, different training regimes reveal a functional specialization akin to the cortex: wav2vec 2.0 learns sound-generic, speech-specific and language-specific representations similar to those of the prefrontal and temporal cortices. Fourth, we confirm the similarity of this specialization with the behavior of 386 additional participants. These elements, resulting from the largest neuroimaging benchmark to date, show how self-supervised learning can account for a rich organization of speech processing in the brain, and thus delineate a path to identify the laws of language acquisition which shape the human brain.
Added
2026-09-26
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, Quoc V. Le
Why you should read this
Introduces an effective data augmentation method that applies time-frequency masking and warping directly to spectrogram features, enabling end-to-end speech recognition models to outperform complex hybrid systems on standard benchmarks.
We present SpecAugment, a simple data augmentation method for speech recognition. SpecAugment is applied directly to the feature inputs of a neural network (i.e., filter bank coefficients). The augmentation policy consists of warping the features, masking blocks of frequency channels, and masking blocks of time steps. We apply SpecAugment on Listen, Attend and Spell networks for end-to-end speech recognition tasks. We achieve state-of-the-art performance on the LibriSpeech 960h and Swichboard 300h tasks, outperforming all prior work. On LibriSpeech, we achieve 6.8% WER on test-other without the use of a language model, and 5.8% WER with shallow fusion with a language model. This compares to the previous state-of-the-art hybrid system of 7.5% WER. For Switchboard, we achieve 7.2%/14.6% on the Switchboard/CallHome portion of the Hub5'00 test set without the use of a language model, and 6.8%/14.1% with shallow fusion, which compares to the previous state-of-the-art hybrid system at 8.3%/17.3% WER.
Added
2026-09-11