Back

EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks

Jason WeiKai Zou

article2019EMNLP2,398 citations

Proposes four simple text data augmentation techniques—synonym replacement, random insertion, random swap, and random deletion—that improve classification performance and allow models trained on half the data to match full-dataset accuracy.

Listen

Text classification models typically require large volumes of labeled training data to achieve high accuracy, yet collecting and labeling this data is often expensive and time-consuming. While other machine learning fields routinely use data augmentation to generate synthetic training examples, natural language processing lacks standardized, easy-to-use augmentation techniques because simple language transformation rules are difficult to generalize.

The article evaluates whether a set of simple, universal text-editing operations—termed Easy Data Augmentation (EDA)—can improve text classification accuracy and reduce overfitting without requiring external datasets or complex deep learning models.

The authors conducted an empirical study evaluating four lightweight operations: synonym replacement, random word insertion, random word swap, and random word deletion. They tested these techniques on five benchmark text classification datasets spanning sentiment analysis, subjectivity, and question categorization, using two standard neural network architectures (recurrent and convolutional neural networks). To measure performance across varying resource levels, the authors trained models using full datasets as well as reduced training subsets ranging from 1% to 100% of available data (specifically focusing on subsets of 500, 2,000, and 5,000 samples).

The evaluation revealed several key findings. First, EDA demonstrated substantial gains for small datasets: models trained on only 500 samples achieved an average accuracy gain of 3.0 percentage points across the benchmark tasks. Second, when using EDA, models trained on just 50% of the available training data matched the average accuracy (88.6%) achieved by baseline models using 100% of the data. Third, while all four operations contributed to performance gains, modifying approximately 10% of the words in a sentence proved to be the optimal parameter across tasks. Latent space visual analysis confirmed that these modest transformations effectively preserve the original class labels of the sentences.

These findings indicate that organizations can significantly cut data collection and labeling costs for text classification initiatives. Because EDA relies solely on basic word manipulations and standard lexical dictionaries, teams can implement data augmentation with minimal computational overhead and zero need to train auxiliary language models. When full datasets are readily available, however, performance improvements from EDA become marginal (an average increase of 0.8%).

For practical application in low-resource settings (such as 500 to 2,000 training examples), practitioners should generate 8 to 16 augmented sentences per original text while altering about 5% to 10% of the words. For larger datasets, practitioners should limit generation to 4 augmented sentences per example to avoid unnecessary compute overhead.

The primary limitation of the study is that EDA offers minimal performance lift when massive training datasets are available or when using modern large, pre-trained language models. Decision-makers can have high confidence in applying EDA as a lightweight, low-cost baseline to boost model robustness in data-constrained text classification scenarios.

Dartmouth CollegeGeorgetown UniversityProtago Labsboostingdata augmentationrecurrent neural networksEDADataData Augmentation
arXiv: 1901.11196jasonwei20/eda_nlp
Cover for EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks

Abstract

We present EDA: easy data augmentation techniques for boosting performance on text classification tasks. EDA consists of four simple but powerful operations: synonym replacement, random insertion, random swap, and random deletion. On five text classification tasks, we show that EDA improves performance for both convolutional and recurrent neural networks. EDA demonstrates particularly strong results for smaller datasets; on average, across five datasets, training with EDA while using only 50% of the available training set achieved the same accuracy as normal training with all available data. We also performed extensive ablation studies and suggest parameters for practical use.

Topics

boostingdata augmentationrecurrent neural networkstext classificationtraining sampletransfer learningclassification models

Keywords

EDADataData AugmentationTextClassi CationPerformanceTasksText Classi CationTechniquesEasy Data Augmentation

Table of Contents

  • 1 Introduction
  • 2 EDA
  • 3 Experimental Setup
  • 3.1 Benchmark Datasets
  • 3.2 Text Classification Models
  • 4 Results
  • 4.1 EDA Makes Gains
  • 4.2 Training Set Sizing
  • 4.3 Does EDA conserve true labels?
  • 4.4 Ablation Study: EDA Decomposed
  • 4.5 How much augmentation?
  • 5 Comparison with Related Work
  • 6 Discussion and Limitations
  • 7 Conclusions
  • 8 Acknowledgements
  • References
  • 9 Supplementary Material
  • 9.1 Implementation Details
  • 9.2 Benchmark Datasets
  • 10 Frequently Asked Questions
  • 10.1 Implementation
  • 10.2 Usage
  • 10.3 Theory

Knowls

  1. Knowl 1 — Easy Data Augmentation (EDA) Operations for Text

    model/method

    Easy Data Augmentation (EDA) consists of four rule-based text editing operations applied to training sentences to generate synthetic text examples without requiring deep generative models or external language models:

    1. Synonym Replacement (SR): Randomly select nn non-stop words from a sentence of length ll words and replace each selected word with a randomly chosen synonym from WordNet.
    2. Random Insertion (RI): Randomly select a non-stop word from the sentence, find a synonym using WordNet, and insert that synonym at a randomly chosen position in the sentence. Repeat this process nn times.
    3. Random Swap (RS): Randomly choose two words in the sentence and swap their positions. Repeat this swap nn times.
    4. Random Deletion (RD): Iterate through each word in the sentence and remove it with independent probability pp.

    To scale the degree of perturbation with sentence length ll, the number of modified words is set to n=αln = \alpha l, and the deletion probability is set to p=αp = \alpha, where α∈[0,1]\alpha \in [0, 1] represents the fraction of words in the sentence targeted for modification. For each original sentence, naugn_{aug} augmented sentences are generated by randomly picking and executing one of the four operations.

  2. Knowl 2 — EDA Text Augmentation Algorithm

    algorithm

    The EDA augmentation procedure generates naugn_{aug} augmented text variants for a given input sentence using a modification parameter α\alpha.

    Input: Sentence SS consisting of words (w1,w2,…,wl)(w_1, w_2, \dots, w_l), modification fraction α\alpha, number of augmentations naugn_{aug}, synonym lexicon (WordNet), stop word list
    Output: List of augmented sentences AA
    Initialize A←∅A \leftarrow \emptyset
    n←round(α⋅l)n \leftarrow \mathrm{round}(\alpha \cdot l)
    p←αp \leftarrow \alpha
    for i=1i = 1 to naugn_{aug} do
        Choose operation op∈{SR,RI,RS,RD}op \in \{\text{SR}, \text{RI}, \text{RS}, \text{RD}\} uniformly at random
        S′←SS' \leftarrow S
        if op==SRop == \text{SR} then
            NonStopWords ←{w∈S′∣w∉stop word list}\leftarrow \{w \in S' \mid w \notin \text{stop word list}\}
            TargetWords ←\leftarrow sample min⁡(n,∣NonStopWords∣)\min(n, |\text{NonStopWords}|) words from NonStopWords without replacement
            for each w∈TargetWordsw \in TargetWords do
                syns←synonyms(w)syns \leftarrow \text{synonyms}(w)
                if syns≠∅syns \neq \emptyset then
                    s←random_choice(syns)s \leftarrow \text{random\_choice}(syns)
                    Replace first occurrence of ww in S′S' with ss
                end if
            end for
        else if op==RIop == \text{RI} then
            NonStopWords ←{w∈S′∣w∉stop word list}\leftarrow \{w \in S' \mid w \notin \text{stop word list}\}
            for k=1k = 1 to nn do
                if NonStopWords ≠∅\neq \emptyset then
                    w←random_choice(NonStopWords)w \leftarrow \text{random\_choice}(\text{NonStopWords})
                    syns←synonyms(w)syns \leftarrow \text{synonyms}(w)
                    if syns≠∅syns \neq \emptyset then
                        s←random_choice(syns)s \leftarrow \text{random\_choice}(syns)
                        pos←random_integer(0,∣S′∣)pos \leftarrow \text{random\_integer}(0, |S'|)
                        Insert ss into S′S' at index pospos
                    end if
                end if
            end for
        else if op==RSop == \text{RS} then
            for k=1k = 1 to nn do
                if ∣S′∣≥2|S'| \ge 2 then
                    idx1,idx2←idx_1, idx_2 \leftarrow sample 2 distinct indices from {1,…,∣S′∣}\{1, \dots, |S'|\}
                    Swap words S′[idx1]S'[idx_1] and S′[idx2]S'[idx_2]
                end if
            end for
        else if op==RDop == \text{RD} then
            FilteredWords←∅FilteredWords \leftarrow \emptyset
            for each w∈S′w \in S' do
                u←random_uniform(0,1)u \leftarrow \text{random\_uniform}(0, 1)
                if u>pu > p then
                    Append ww to FilteredWordsFilteredWords
                end if
            end for
            if FilteredWords≠∅FilteredWords \neq \emptyset then
                S′←FilteredWordsS' \leftarrow FilteredWords
            end if
        end if
        Append S′S' to AA
    end for
    return AA

  3. Knowl 3 — Text Classification Performance Across Training Set Sizes With and Without EDA

    data/table

    The performance of recurrent neural networks (LSTM-RNN) and convolutional neural networks (CNN) was evaluated on five benchmark text classification datasets (SST-2 sentiment analysis, Customer Reviews [CR], Subjectivity/Objectivity [SUBJ], TREC question classification, and Pro-Con [PC]) across training set sample sizes Ntrain∈{500,2000,5000,full set}N_{train} \in \{500, 2000, 5000, \text{full set}\}. Results represent average test accuracy (%) over five random seeds.

    Model 500 2,000 5,000 full set
    RNN 75.3 83.7 86.1 87.4
    +EDA 79.1 84.4 87.3 88.3
    CNN 78.6 85.6 87.7 88.3
    +EDA 80.7 86.4 88.3 88.8
    Average 76.9 84.6 86.9 87.8
    +EDA 79.9 85.4 87.8 88.6

    EDA consistently improves classification performance across all training set sizes and both architectures. The relative gain is most pronounced in low-resource regimes, yielding a +3.0%+3.0\% absolute accuracy improvement at Ntrain=500N_{train}=500, compared to a +0.8%+0.8\% gain when trained on the full dataset.

  4. Knowl 4 — Data Efficiency Equivalence Between 50% Augmented Data and Full Dataset Training

    empirical result

    Across five text classification benchmarks (SST-2, CR, SUBJ, TREC, and Pro-Con), training text classification models (CNNs and LSTMs) using EDA on only 50%50\% of the available training data achieves an average classification accuracy of 88.6%88.6\%. This matches and slightly outperforms the baseline performance of standard training without augmentation using 100%100\% of the available training data (88.3%88.3\%).

  5. Knowl 5 — Sensitivity and Performance Gain of Individual EDA Operations Across Modification Fraction $\alpha$

    empirical result

    When isolated individually across five text classification tasks, all four EDA operations (Synonym Replacement, Random Insertion, Random Swap, Random Deletion) produce positive performance gains for small values of the word modification parameter α∈[0.05,0.2]\alpha \in [0.05, 0.2]:

    • Synonym Replacement (SR): Yields high performance improvements at α≤0.1\alpha \le 0.1, but degrades at α≥0.3\alpha \ge 0.3 because replacing excessive words alters the core semantic meaning of the sentence.
    • Random Insertion (RI): Maintains stable performance gains across α∈[0.05,0.5]\alpha \in [0.05, 0.5] because original words and their sequential ordering remain intact.
    • Random Swap (RS): Delivers strong performance gains at α≤0.2\alpha \le 0.2, but accuracy deteriorates sharply at α≥0.3\alpha \ge 0.3 where excessive position swapping approaches completely shuffling the sentence.
    • Random Deletion (RD): Produces the highest performance gains at low α\alpha (e.g., α=0.05\alpha = 0.05 to 0.10.1), but severely degrades classification accuracy at high α\alpha (e.g., α≥0.4\alpha \ge 0.4) as omitting substantial proportions of words renders sentences unintelligible.

    Across all operations and dataset sizes, α=0.1\alpha = 0.1 serves as an optimal modification rate.

  6. Knowl 6 — Recommended EDA Hyperparameters Based on Training Set Size

    data/table

    The optimal word modification rate α\alpha and number of generated augmented sentences per original sample naugn_{aug} depend on the size of the training dataset NtrainN_{train}. Smaller datasets benefit from a higher number of generated sentences to combat overfitting, whereas larger datasets require fewer augmentations to prevent redundancy.

    NtrainN_{train} α\alpha naugn_{aug}
    500 0.05 16
    2,000 0.05 8
    5,000 0.1 4
    >5,000> 5,000 0.1 4

  7. Knowl 7 — Latent Space Preservation of True Class Labels Under EDA Operations

    empirical result

    To evaluate whether EDA text transformations alter semantic identity and invalidate original ground-truth labels, nine augmented sentences per test example were generated on the Pro-Con classification task and fed into an unaugmented pre-trained LSTM-RNN classifier. Two-dimensional t-SNE projections of representations extracted from the model's final dense layer demonstrated that augmented sentence embeddings closely cluster around the latent space embeddings of their corresponding unaugmented seed sentences, indicating that EDA operations preserve ground-truth class labels.

  8. Knowl 8 — Limitations of EDA on Large Datasets and Pre-trained Language Models

    limitation

    EDA demonstrates two primary operational limitations:

    1. Diminishing Returns on Large Datasets: When the volume of labeled training data is large, the performance boost from EDA is marginal, achieving an average accuracy gain of less than 1.0%1.0\% when full datasets (Ntrain>5,000N_{train} > 5,000) are utilized.
    2. Limited Efficacy with Pre-trained Contextual Embeddings: While EDA improves standard neural architectures (CNNs and LSTM-RNNs) trained from scratch, empirical evidence shows negligible performance gains when applied in conjunction with large pre-trained contextualized language models (such as ULMFiT, ELMo, or BERT).

Coverage note — None was omitted; all key contributions including the four editing operations, algorithm formulation, empirical evaluation across benchmarks and architectures, ablation studies, hyperparameter recommendations, latent space analysis, and documented limitations are fully covered.

References

  1. 1.Xiaodong Cui, Vaibhava Goel, and Brian Kingsbury. 2015. Data augmentation for deep neural network acoustic modeling. IEEE/ACM Trans. Audio, Speech and Lang. Proc., 23(9):1469–1477.
  2. 2.Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
  3. 3.Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. Data augmentation for low-resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 567–573. Association for Computational Linguistics.
  4. 4.Murthy Ganapathibhotla and Bing Liu. 2008. Mining opinions in comparative sentences. In Proceedings of the 22Nd International Conference on Computational Linguistics - Volume 1, COLING ’08, pages 241–248, Stroudsburg, PA, USA. Association for Computational Linguistics.
  5. 5.Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177, New York, NY, USA. ACM.
  6. 6.Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward controlled generation of text. In ICML.
  7. 7.Kushal Kafle, Mohammed Yousefhussien, and Christopher Kanan. 2017. Data augmentation for visual question answering. In Proceedings of the 10th International Conference on Natural Language Generation, pages 198–202. Association for Computational Linguistics.
  8. 8.Yoon Kim. 2014. Convolutional neural networks for sentence classification. CoRR, abs/1408.5882.
  9. 9.Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In INTERSPEECH.
  10. 10.Sosuke Kobayashi. 2018. Contextual augmentation: Data augmentation by words with paradigmatic relations. In NAACL-HLT.
  11. 11.Oleksandr Kolomiyets, Steven Bethard, and Marie-Francine Moens. 2011. Model-portability experiments for textual temporal analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 271–276, Stroudsburg, PA, USA. Association for Computational Linguistics.
  12. 12.Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM, 60(6):84–90.
  13. 13.Xin Li and Dan Roth. 2002. Learning question classifiers. In Proceedings of the 19th International Conference on Computational Linguistics - Volume 1, COLING ’02, pages 1–7, Stroudsburg, PA, USA. Association for Computational Linguistics.
  14. 14.Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Recurrent neural network for text classification with multi-task learning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pages 2873–2879. AAAI Press.
  15. 15.Qian Liu, Zhiqiang Gao, Bing Liu, and Yuanlin Zhang. 2015. Automated rule selection for aspect extraction in opinion mining. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pages 1291–1297. AAAI Press.
  16. 16.George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39–41.
  17. 17.Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42Nd Annual Meeting on Association for Computational Linguistics, ACL ’04, Stroudsburg, PA, USA. Association for Computational Linguistics.
  18. 18.Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543.
  19. 19.Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. CoRR, abs/1802.05365.
  20. 20.David Rolnick, Andreas Veit, Serge J. Belongie, and Nir Shavit. 2017. Deep learning is robust to massive label noise. CoRR, abs/1705.10694.
  21. 21.Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96. Association for Computational Linguistics.
  22. 22.Sam Shleifer. 2019. Low resource text classification with ulmfit and backtranslation. CoRR, abs/1903.09244.
  23. 23.Miikka Silfverberg, Adam Wiemerslage, Ling Liu, and Lingshuang Jack Mao. 2017. Data augmentation for morphological reinflection. In Proceedings of the CoNLL SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection, pages 90–99. Association for Computational Linguistics.
  24. 24.Patrice Simard, Yann LeCun, John S. Denker, and Bernard Victorri. 1998. Transformation invariance in pattern recognition-tangent distance and tangent propagation. In Neural Networks: Tricks of the Trade, This Book is an Outgrowth of a 1996 NIPS Workshop, pages 239–27, London, UK, UK. Springer-Verlag.
  25. 25.Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. 2013. Parsing With Compositional Vector Grammars. In EMNLP.
  26. 26.Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2014. Going deeper with convolutions. CoRR, abs/1409.4842.
  27. 27.Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. pages 1422–1432.
  28. 28.Simon Tong and Daphne Koller. 2002. Support vector machine active learning with applications to text classification. J. Mach. Learn. Res., 2:45–66.
  29. 29.Laurens Van Der Maaten. 2014. Accelerating t-sne using tree-based algorithms. J. Mach. Learn. Res., 15(1):3221–3245.
  30. 30.William Yang Wang and Diyi Yang. 2015. That’s so annoying!!!: A lexical and frame-semantic embedding based data augmentation approach to automatic categorization of annoying behaviors using #petpeeve tweets. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2557–2563. Association for Computational Linguistics.
  31. 31.Ziang Xie, Sida I. Wang, Jiwei Li, Daniel Levy, Aiming Nie, Dan Jurafsky, and Andrew Y. Ng. 2017. Data noising as smoothing in neural network language models.
  32. 32.Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. 2018. Qanet: Combining local convolution with global self-attention for reading comprehension. CoRR, abs/1804.09541.
  33. 33.Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, pages 649–657, Cambridge, MA, USA. MIT Press.

Citation

MLA
Wei, J., and K. Zou. “EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks”. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 6381–87, https://doi.org/10.18653/v1/D19-1670.
APA
Wei, J., & Zou, K. (2019). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 6381–6387. https://doi.org/10.18653/v1/D19-1670
Chicago
Wei, J., and K. Zou. 2019. “EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks”. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 6381–87. https://doi.org/10.18653/v1/D19-1670.
Harvard
Wei, J. and Zou, K. (2019) “EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks”, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, pp. 6381–6387. Available at: https://doi.org/10.18653/v1/D19-1670.
Vancouver
1. Wei J, Zou K (2019) EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, pp 6381–6387

BibTeX

@inproceedings{Wei_2019, title={EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks}, url={http://dx.doi.org/10.18653/v1/D19-1670}, DOI={10.18653/v1/d19-1670}, booktitle={Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)}, publisher={Association for Computational Linguistics}, author={Wei, Jason and Zou, Kai}, year={2019}, pages={6381–6387} }
Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/