Common Voice: A Massively-Multilingual Speech Corpus

Rosana ArdilaMegan BransonKelly DavisMichael HenrettyMichael KohlerJosh MeyerReuben MoraisLindsay SaundersFrancis M. TyersGregor Weber

article2019LREC2,600 citations

Introduces Common Voice, a massively multilingual and publicly accessible speech corpus of over 2,500 hours across dozens of languages, and establishes baseline transfer-learning results that yield the first published end-to-end speech recognition benchmarks for several under-resourced languages.

Listen

The article addresses the scarcity of affordable or available training data for speech recognition technologies across most languages, which limits the development of accessible speech tools worldwide.

This work set out to create and evaluate a large-scale, crowdsourced multilingual speech corpus called Common Voice, along with demonstrating its utility through automatic speech recognition experiments.

Researchers built the corpus using a web platform and app where volunteers record and validate speech clips in their languages. Data from 29 languages totaling over 2,500 hours was collected by more than 50,000 participants as of late 2019. Experiments applied transfer learning from a pre-trained English model to twelve target languages using Mozilla’s DeepSpeech toolkit.

The analysis shows an average Character Error Rate improvement of about 6 points across the twelve languages when four layers were transferred from the English model. Several languages received their first published end-to-end speech recognition results. The corpus is the largest public-domain collection by both hours and languages.

These results mean organizations can now access open data to build or improve speech systems for low-resource languages without high licensing costs or proprietary restrictions. Performance gains reduce the barrier for new language support and support broader deployment of voice interfaces.

Language communities should continue translating the interface, collecting sentences, and recording voices to expand coverage. Additional validation and data splits will strengthen future releases.

The findings rest on an internal data snapshot and early validation mechanics that contained bugs; some languages still have very small speaker counts, so results should be treated as promising but preliminary until larger, cleaner releases are tested.

Artie, Inc.Indiana University Mozilla audio automatic speech recognition language identification Common Voice corpus low-resource languages speech recognition

arXiv: 1912.06670

Paper: Deep Speech: Scaling up end-to-end speech recognition, Awni Y. Hannun et al. (2014). It introduces the foundational Deep Speech end-to-end recurrent architecture and CTC pipeline that Mozilla's DeepSpeech framework and the Common Voice experiments directly build upon.
Paper: Deep Speech 2: End-to-End Speech Recognition in English and Mandarin, Dario Amodei et al. (2016). It details the scaled multi-language end-to-end ASR architecture and optimization practices that established the viability of cross-lingual deep speech modeling.
Paper: Towards End-To-End Speech Recognition with Recurrent Neural Networks, Alex Graves et al. (2014). It establishes foundational CTC-based recurrent neural network training for speech-to-text without intermediate phonetic alignments.
Paper: Speech Recognition with Deep Recurrent Neural Networks, Alex Graves et al. (2013). It provides the core theoretical foundation for applying deep bidirectional recurrent neural networks directly to acoustic sequence processing.

Paper: Robust Speech Recognition via Large-Scale Weak Supervision, Alec Radford et al. (2023). It evaluates massively multilingual speech recognition at internet scale using weak supervision and benchmarks across diverse public datasets like Common Voice.
Paper: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations, Alexei Baevski et al. (2020). It provides a self-supervised representation learning framework that drastically reduces the labeled data required for low-resource automatic speech recognition.
Paper: HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units, Wei-Ning Hsu et al. (2021). It introduces masked hidden-unit prediction to advance self-supervised acoustic representation learning for low-resource speech recognition.
Paper: Conformer: Convolution-augmented Transformer for Speech Recognition, Anmol Gulati et al. (2020). It develops the Conformer architecture combining convolutions and self-attention, significantly advancing acoustic modeling performance beyond earlier recurrent models.
Paper: WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing, Sanyuan Chen et al. (2021). It scales joint masked speech prediction and denoising to create general-purpose acoustic representations across diverse speech processing tasks.

Cover for Common Voice: A Massively-Multilingual Speech Corpus

Abstract

The Common Voice corpus is a massively-multilingual collection of transcribed speech intended for speech technology research and development. Common Voice is designed for Automatic Speech Recognition purposes but can be useful in other domains (e.g. language identification). To achieve scale and sustainability, the Common Voice project employs crowdsourcing for both data collection and data validation. The most recent release includes 29 languages, and as of November 2019 there are a total of 38 languages collecting data. Over 50,000 individuals have participated so far, resulting in 2,500 hours of collected audio. To our knowledge this is the largest audio corpus in the public domain for speech recognition, both in terms of number of hours and number of languages. As an example use case for Common Voice, we present speech recognition experiments using Mozilla's DeepSpeech Speech-to-Text toolkit. By applying transfer learning from a source English model, we find an average Character Error Rate improvement of 5.99 +/- 5.48 for twelve target languages (German, French, Italian, Turkish, Catalan, Slovenian, Welsh, Irish, Breton, Tatar, Chuvash, and Kabyle). For most of these languages, these are the first ever published results on end-to-end Automatic Speech Recognition.

Topics

audio automatic speech recognition language identification speech signals transfer learning validation speech synthesis speech recognition systems speech data audio stream

Keywords

Common Voice corpus low-resource languages speech recognition transfer learning Connectionist Temporal Classification Word Error Rate Character Error Rate

1. Introduction
2. Prior work
3. Corpus Creation
4. Corpus Contents
4.1. Released Languages
4.2. Adding a new Language
5. Automatic Speech Recognition Experiments
5.1. Data
5.2. Model architecture
6. Results
7. Concluding remarks
8. References
References

Knowls

Knowl 1 — Common Voice Crowdsourced Recording and Validation Protocol
model/method
The Common Voice corpus relies on a crowdsourced collection and validation pipeline accessible via web and mobile interfaces:
1. Voice Recording: Contributors read displayed text prompts aloud to generate audio clips.
2. Voice Validation: Independent contributors listen to recorded $\langle\text{audio}, \text{transcript}\rangle$ pairs and submit binary up-votes (correct) or down-votes (incorrect).
3. Validation Decision Rule: A maximum of three contributors review each clip:
  - If a clip accumulates two up-votes first, it is categorized as valid.
  - If a clip accumulates two down-votes first, it is categorized as invalid.
  - Clips that have not accumulated sufficient votes at dataset release time are categorized as other.
Only validated clips are bucketed into official training, development, and test sets. All audio and transcript data are released openly under the Creative Commons CC0 public domain dedication.

Knowl 2 — Transfer Learning Character Error Rate Across Languages by Number of Transferred Layers

empirical result

Fine-tuning acoustic models initialized with layers from a pre-trained English DeepSpeech v0.3.0 model improves downstream Character Error Rate (CER, computed as the Levenshtein distance between ground-truth and decoded character sequences) across 12 target languages compared to training from scratch (0 transferred layers):

Language	None (0)	1 Layer	2 Layers	3 Layers	4 Layers	5 Layers
Slovenian (sl)	23.35	21.65	26.44	19.09	15.35	17.96
Irish (ga)	31.83	31.01	32.20	27.50	25.42	24.98
Chuvash (cv)	48.10	47.10	44.58	42.75	27.21	31.94
Breton (br)	21.47	19.16	20.01	18.06	15.99	18.42
Turkish (tr)	34.66	34.12	34.83	31.79	27.55	29.74
Italian (it)	40.91	42.65	42.82	36.89	33.63	35.10
Welsh (cy)	34.15	31.91	33.63	30.13	28.75	30.38
Tatar (tt)	32.61	31.43	30.80	27.79	26.42	28.63
Catalan (ca)	38.01	35.21	39.02	35.26	33.83	36.41
French (fr)	43.33	43.26	43.51	43.24	43.20	43.19
Kabyle (kab)	25.76	25.50	26.83	25.25	24.92	25.28
German (de)	43.76	43.69	43.62	43.60	43.76	43.69

Key findings:

Transferring 4 layers (the initial three fully connected layers plus the LSTM layer) achieves the best performance for 8 of the 12 languages (sl, cv, br, tr, it, cy, tt, kab).
Averaged across all 12 target languages, transferring 4 layers achieves an absolute CER reduction of $5.99 \pm 5.48$ percentage points relative to training from scratch.
The performance gain is substantial on low-resource datasets (e.g., Chuvash improves by $20.89$ CER points; Slovenian improves by $8.00$ CER points), whereas for high-resource datasets (French and German with $>40,000$ training clips), transfer learning yields negligible difference ( $<0.2$ CER change).

Knowl 3 — DeepSpeech Transfer Learning ASR Architecture and Training Setup
experimental setup
Speech recognition experiments on the Common Voice dataset are implemented using Mozilla's DeepSpeech v0.3.0, an end-to-end Automatic Speech Recognition architecture trained via Connectionist Temporal Classification (CTC) loss:
- Input Features: Acoustic input is represented by splicing 19 temporal frames (9 past frames, 1 center frame, 9 future frames) with 26 Mel-Frequency Cepstral Coefficients (MFCC) per frame, producing a 494-dimensional feature vector per time step.
- Network Topology: A 6-layer neural network comprising:
  - Layers 1--3: Fully connected feedforward layers with 2,048 hidden units each and clipped Rectified Linear Unit (clipped $\text{ReLU}$ ) activations.
  - Layer 4: Unidirectional Long Short-Term Memory (LSTM) layer with 2,048 units.
  - Layer 5: Fully connected feedforward layer with 2,048 units and clipped $\text{ReLU}$ activation.
  - Layer 6 (Output): Linear softmax projection layer with dimensionality equal to the target language alphabet size plus punctuation and the CTC blank symbol.
- Training and Optimization:
  - Optimizer: Adam with a learning rate of $0.0001$ .
  - Batch sizes: 24 for training, 48 for validation.
  - Regularization: Dropout rate of $20\%$ .
  - Layer Initialization: Target-specific layers are initialized with Xavier initialization.
  - Fine-Tuning: Copied pre-trained layers from an English source model and newly initialized layers are fine-tuned jointly via gradient descent.
- Early Stopping: Backpropagation halts when development set loss either (1) increases across 5 sequential epochs, or (2) fails to improve by more than a mean threshold of 0.5 over a 5-epoch window with a standard deviation below 0.5.
Knowl 4 — Dataset Partitioning and Speaker-Disjoint Splitting Protocol in Common Voice
model/method
To evaluate ASR models on true speaker generalization without data leakage, Common Voice partitions validated clips into training, development, and test splits following three rules:
1. Speaker Disjointness: A unique speaker identifier (client_id) appears exclusively within a single partition (train, development, or test).
2. Sentence Deduplication: Exact duplicate text prompts are eliminated across the train, development, and test splits.
3. Power Analysis Sizing: For a given language, test and development set sizes are statistically calculated such that each split achieves a $99\%$ confidence level with a $1\%$ margin of error relative to the training set.
Knowl 5 — Onboarding Pipeline and Sentence Sourcing for New Languages in Common Voice
model/method
Integrating a new language into Common Voice requires two distinct milestones:
1. User Interface Localization: Community volunteers translate 610 user interface strings via the Pontoon localization platform. Translations are reviewed and approved by language-specific moderators.
2. Text Prompt Collection:
  - For languages containing more than 500,000 Wikipedia articles, candidate sentences are extracted from Wikipedia using community-defined rule sets.
  - For other languages or supplementary gathering, sentences are submitted through the Sentence Collector tool, which runs automated validation checks (length, foreign character sets, and numeric digits) and requires approval by two out of three independent reviewers.
3. Recording Enablement: Once the web interface is fully localized and a minimum of 5,000 valid text sentences have been ingested, recording is activated for the language.
Knowl 6 — Common Voice Audio Format and Metadata Schema
definition
Common Voice releases speech audio and metadata under the following specification:
- Audio Format: Mono-channel, 16-bit MPEG-3 files sampled at $48\text{ kHz}$ . MPEG-3 is used to ensure maximum compatibility for browser- and mobile-based recording and playback.
- Metadata Structure: Each language release includes tab-separated values (TSV) files containing the following 8 fields:
  - client_id: Anonymized unique contributor identifier.
  - path: Audio file path.
  - sentence: Text transcription corresponding to the recording.
  - up_votes: Total count of positive validation votes.
  - down_votes: Total count of negative validation votes.
  - age: Self-reported speaker age category (optional).
  - gender: Self-reported speaker gender (optional).
  - accent: Self-reported speaker accent (optional).

Knowl 7 — Common Voice Corpus Statistics (June 2019 Release)

data/table

The Common Voice corpus comprises 58,250 individual contributors and 2,508 total hours of speech across 38 languages (2,019 hours validated as of the June 12, 2019 release and active pipelines):

Language	Code	Voices	Total Hours	Validated Hours
Abkhaz	ab	3	$<1$	$<1$
Arabic	ar	225	15	9
Basque	eu	508	83	46
Breton	br	118	10	3
Catalan	ca	1,834	120	107
Chinese (China)	zh-ZH	288	12	11
Chinese (Taiwan)	zh-TW	949	43	33
Chuvash	cv	38	2	1
Dhivehi	dv	92	8	5
Dutch	nl	502	23	18
English	en	39,577	1,087	780
Esperanto	eo	129	16	13
Estonian	et	225	12	11
French	fr	3,005	184	173
German	de	5,007	340	325
Hakha Chin	cnh	280	4	2
Indonesian	id	54	5	4
Interlingua	ia	11	2	1
Irish	ga	63	3	2
Italian	it	602	40	36
Japanese	ja	48	2	1
Kabyle	kab	584	192	181
Kinyarwanda	rw	32	1	$<1$
Kyrgyz	ky	97	20	8
Latvian	lv	82	8	6
Mongolian	mn	230	9	8
Persian	fa	1,240	70	67
Portuguese	pr	316	30	27
Russian	ru	64	31	27
Sakha	sah	35	6	3
Slovenian	sl	42	5	2
Spanish	es	611	31	27
Swedish	sv	44	3	3
Tamil	ta	89	5	3
Tatar	tt	132	26	22
Turkish	tr	344	10	9
Votic	vot	2	$<1$	$<1$
Welsh	cy	748	48	42
TOTAL		58,250	2,508	2,019

The dataset covers major global languages alongside low-resource and regional languages, with English being the largest subset (39,577 voices, 1,087 hours), followed by German (5,007 voices, 340 hours) and French (3,005 voices, 184 hours).

Knowl 8 — Experimental Audio Clip and Speaker Distribution for 12 Target Languages

data/table

The transfer learning ASR experiments evaluated models across 12 target languages using an internal February 2019 release split into speaker-disjoint partitions targeting approximately $80\%$ train, $10\%$ dev, and $10\%$ test distributions:

		Audio Clips			Unique Speakers
Language	Code	Dev	Test	Train	Dev	Test	Train
Slovenian	sl	110	213	728	1	12	3
Irish	ga	181	138	1,001	4	12	6
Chuvash	cv	96	77	1,023	4	12	5
Breton	br	163	170	1,079	3	15	7
Turkish	tr	407	374	3,771	32	89	32
Italian	it	627	734	5,019	29	136	37
Welsh	cy	1,235	1,201	9,547	51	153	75
Tatar	tt	1,811	1,164	11,187	9	64	3
Catalan	ca	5,460	5,037	38,995	286	777	313
French	fr	5,083	4,835	40,907	237	837	249
Kabyle	kab	5,452	4,643	43,223	31	169	63
German	de	7,982	7,897	65,745	247	1,029	318

Enforcing strict speaker disjointness creates challenging low-speaker evaluation conditions for several low-resource languages (e.g., Slovenian and Tatar each have only 3 distinct training speakers).

Coverage note — No substantial contributed material was omitted.

References

1.Fiscus, J. G., Ajot, J., Radde, N., and Laprun, C. (2006). Multiple dimension levenshtein edit distance calculations for evaluating automatic speech recognition systems during simultaneous speech. In LREC, pages 803–808. Citeseer.
2.Gales, M. J., Knill, K. M., Ragni, A., and Rath, S. P. (2014). Speech recognition and keyword spotting for low-resource languages: Babel project research at cued. In Spoken Language Technologies for Under-Resourced Languages.
3.Glorot, X. and Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256.
4.Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J. (2006). Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369–376. ACM.
5.Hannun, A. Y., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., Coates, A., and Ng, A. Y. (2014). Deep Speech: Scaling up end-to-end speech recognition. CoRR, abs/1412.5567.
6.M-AILABS. (2019). The m-ailabs speech dataset. https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/. accessed 11/25/2019.
7.Meyer, J. (2019). Multi-Task and Transfer Learning in Low-Resource Speech Recognition. PhD dissertation, The University of Arizona.
8.Roter, G. (2019). Sharing our common voices mozilla releases the largest to-date public domain transcribed voice dataset, Feb. https://blog.mozilla.org/blog/2019/02/28/sharing-our-common-voices-mozilla-releases-the-largest-to-date-public-domain-transcribed-voice-dataset/.
9.VoxForge. (2019). Voxforge. http://www.voxforge.org/. accessed 11/25/2019.

Citation

MLA

Ardila, R., et al. “Common Voice: A Massively-Multilingual Speech Corpus”. arXiv, 2019, https://doi.org/10.48550/arxiv.1912.06670.

APA

Ardila, R., Branson, M., Davis, K., Henretty, M., Kohler, M., Meyer, J., Morais, R., Saunders, L., Tyers, F. M., & Weber, G. (2019). Common Voice: A Massively-Multilingual Speech Corpus. arXiv. https://doi.org/10.48550/arxiv.1912.06670

Chicago

Ardila, R., M. Branson, K. Davis, et al. 2019. “Common Voice: A Massively-Multilingual Speech Corpus”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1912.06670.

Harvard

Ardila, R. et al. (2019) “Common Voice: A Massively-Multilingual Speech Corpus”. arXiv. Available at: https://doi.org/10.48550/arxiv.1912.06670.

Vancouver

1. Ardila R, Branson M, Davis K, Henretty M, Kohler M, Meyer J, Morais R, Saunders L, Tyers FM, Weber G (2019) Common Voice: A Massively-Multilingual Speech Corpus. https://doi.org/10.48550/arxiv.1912.06670

BibTeX

@misc{https://doi.org/10.48550/arxiv.1912.06670,
  doi = {10.48550/ARXIV.1912.06670},
  url = {https://arxiv.org/abs/1912.06670},
  author = {Ardila, Rosana and Branson, Megan and Davis, Kelly and Henretty, Michael and Kohler, Michael and Meyer, Josh and Morais, Reuben and Saunders, Lindsay and Tyers, Francis M. and Weber, Gregor},
  keywords = {Computation and Language (cs.CL), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Common Voice: A Massively-Multilingual Speech Corpus},
  publisher = {arXiv},
  year = {2019},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Common Voice: A Massively-Multilingual Speech Corpus

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Common Voice Crowdsourced Recording and Validation Protocol

Knowl 2 — Transfer Learning Character Error Rate Across Languages by Number of Transferred Layers

Knowl 3 — DeepSpeech Transfer Learning ASR Architecture and Training Setup

Knowl 4 — Dataset Partitioning and Speaker-Disjoint Splitting Protocol in Common Voice