Back

A systematic study of the class imbalance problem in convolutional neural networks

Mateusz BudaAtsuto MakiMaciej A. Mazurowski

article2017Neural Networks3,128 citations

Demonstrates that complete oversampling consistently outperforms undersampling and thresholding for class-imbalanced convolutional neural networks across standard vision benchmarks without causing the overfitting typical in classical machine learning.

Listen

Class imbalance in training data for convolutional neural networks is a widespread issue in applications such as medical imaging, fraud detection, and object recognition, where one class may appear hundreds or thousands of times more often than another. Prior work had shown that imbalance harms classical machine learning models, yet no large-scale, controlled study existed for modern deep networks.

This paper set out to measure how different degrees and forms of imbalance affect CNN performance and to compare the main practical remedies in a consistent way across tasks of increasing difficulty.

The authors created controlled imbalanced versions of three standard benchmarks—MNIST, CIFAR-10, and ImageNet—using two representative imbalance patterns and trained thousands of networks with LeNet, All-CNN, and ResNet architectures. They evaluated random oversampling, random undersampling, two-phase training, and simple thresholding, using multi-class ROC AUC as the primary metric because overall accuracy can be misleading under imbalance.

The experiments produced five main results. First, imbalance caused clear, substantial drops in performance, and the harm grew markedly with task complexity. Second, oversampling consistently ranked highest across nearly all conditions and never produced the overfitting previously observed in classical models. Third, full oversampling to equal class sizes was optimal, whereas the best undersampling level depended on the fraction of minority classes. Fourth, thresholding improved the raw count of correct predictions when applied after oversampling or on the original data, but it did not change discrimination ability. Fifth, the performance loss could not be explained merely by fewer total examples; the uneven distribution itself mattered.

These findings indicate that practitioners facing typical imbalance can improve results reliably and at modest cost by applying oversampling, while reserving undersampling for cases where training speed is critical and using thresholding when the goal is to maximize the absolute number of correct decisions.

The main limitations are that the largest ImageNet experiments showed run-to-run variability and that extreme imbalance on very large tasks may require further tuning of training schedules. Within the scope examined, however, the ranking of methods is stable and the evidence for oversampling is strong.

Duke UniversityKTH Royal Institute of Technologyclassification methodsconvolutional neural networksdeep learningclass imbalance problemconvolutional neural networkscost-sensitive learning
arXiv: 1710.05381
  • Paper: SMOTE: Synthetic Minority Over-sampling Technique, N. Chawla et al. (2002). Reading this foundational paper on SMOTE provides the key classical data-synthesis concepts that the source paper evaluates and contrasts with deep learning methods.
  • Paper: The foundations of cost-sensitive learning, Charles Elkan (2001). This paper establishes the theoretical foundations of cost-sensitive learning and probability threshold adjustments, directly informing the thresholding techniques evaluated in the source.
  • Paper: Class-Balanced Loss Based on Effective Number of Samples, Yin Cui et al. (2019). This work extends the source study by developing a sophisticated class-balanced loss based on the effective number of samples rather than standard oversampling or undersampling.
  • Paper: Focal Loss for Dense Object Detection, Tsung-Yi Lin et al. (2017). This paper builds directly upon the class imbalance problem investigated in the source by introducing the Focal Loss to address extreme foreground-background imbalance in object detection.
Cover for A systematic study of the class imbalance problem in convolutional neural networks

Abstract

In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.

Topics

classification methodsconvolutional neural networksdeep learningthresholdingclassifiersprobability distributionstransfer learning

Keywords

class imbalance problemconvolutional neural networkscost-sensitive learningimbalance ratiooptimal undersampling ratiomulticlass classificationimage classificationstep imbalancelinear imbalance

Table of Contents

  • A systematic study of the class imbalance problem in convolutional neural networks*
  • 1 Introduction
  • 2 Methods for addressing imbalance
  • 2.1 Data level methods
  • 2.2 Classifier level methods
  • 3 Experiments
  • 3.1 Forms of imbalance
  • 3.2 Methods of addressing imbalance compared in this study
  • 3.3 Datasets and models
  • 3.3.1 MNIST
  • 3.3.2 CIFAR-10
  • 3.3.3 ImageNet
  • 3.4 Evaluation metrics and testing
  • 4 Results
  • 4.1 Effects of class imbalance on classification performance and comparison of methods to address imbalance
  • 4.2 Results on ImageNet dataset
  • 4.3 Separation of effects from reduced number of examples and class imbalance
  • 4.4 Improving accuracy score with multi-class thresholding
  • 4.5 Undersampling and oversampling to smaller imbalance ratio
  • 4.6 Generalization of sampling methods
  • 5 Conclusions
  • Appendix A. Supplementary data
  • References

Knowls

  1. Knowl 1 — Dominance of Random Minority Oversampling for CNN Class Imbalance

    empirical result

    Across extensive experiments on MNIST, CIFAR-10, and ImageNet datasets under varying degrees and forms of class imbalance, random minority oversampling emerged as the consistently superior strategy for training convolutional neural networks (CNNs), as measured by multi-class receiver operating characteristic area under the curve (ROC AUC). Oversampling substantially improves classification performance compared to the baseline (training directly on imbalanced data without intervention) across virtually all evaluated scenarios and never causes notable performance degradation. In contrast, random majority undersampling frequently underperforms baseline training due to discarding valuable training data, achieving parity with oversampling only under extreme imbalance ratios combined with high proportions of minority classes.

  2. Knowl 2 — Absence of Overfitting in Convolutional Neural Networks under Oversampling

    empirical result

    Contrary to classical machine learning models where replicating minority class instances leads to severe overfitting on minority examples, random minority oversampling does not cause overfitting when training convolutional neural networks (CNNs). Across evaluations on MNIST, CIFAR-10 (using the All-CNN architecture), and ImageNet (using ResNet-10), monitoring training and test set accuracy curves across training iterations revealed that the generalization gap between train and test accuracy does not increase during oversampling compared to baseline or undersampled training. Instead, oversampling improves optimization stability and yields superior generalization on balanced test sets.

  3. Knowl 3 — Formal Parametrization of Multiclass Step Imbalance and Linear Imbalance

    definition

    In multiclass classification with NN total classes and class sample subsets C1,C2,…,CNC_1, C_2, \dots, C_N, class imbalance is formalized into two representative profiles:

    1. Step imbalance: Classes are divided into majority and minority groups. All majority classes contain an identical number of samples, and all minority classes contain an identical smaller number of samples. This profile is governed by two parameters:

      • Minority class fraction: μ=∣{i∈{1,…,N}:Ci is minority}∣N\mu = \frac{|\{i \in \{1, \dots, N\} : C_i \text{ is minority}\}|}{N} where μ∈(0,1)\mu \in (0, 1).
      • Imbalance ratio: ρ=max⁡i∣Ci∣min⁡i∣Ci∣\rho = \frac{\max_{i} |C_i|}{\min_{i} |C_i|} where ρ≥1\rho \ge 1.

    2. Linear imbalance: Class sizes are linearly interpolated between the maximum and minimum class sizes such that the difference in sample counts between consecutive sorted classes is constant. This profile is parameterized solely by the imbalance ratio ρ=max⁡i∣Ci∣min⁡i∣Ci∣\rho = \frac{\max_{i} |C_i|}{\min_{i} |C_i|}.

  4. Knowl 4 — Output Thresholding via Bayesian Prior Probability Compensation

    model/method

    Convolutional neural networks trained with cross-entropy loss output predicted posterior class probabilities yi(x)=p(i∣x)=p(i)⋅p(x∣i)p(x)y_i(x) = p(i \mid x) = \frac{p(i) \cdot p(x \mid i)}{p(x)} for an input sample xx and class i∈{1,…,N}i \in \{1, \dots, N\}. When trained on class-imbalanced data, the model implicitly learns the skewed training priors p(i)p(i).

    To adjust decision boundaries post-training without modifying the network weights, thresholding (post-scaling) compensates for the prior class distribution. The estimated prior class probability is: p(i)=∣Ci∣∑k=1N∣Ck∣p(i) = \frac{|C_i|}{\sum_{k=1}^N |C_k|} where ∣Ci∣|C_i| denotes the number of unique training examples in class ii. The network output for class ii is updated by dividing by its estimated prior p(i)p(i) before selecting the predicted class via argmax⁡\operatorname{argmax}: yi′(x)=yi(x)p(i)y_i'(x) = \frac{y_i(x)}{p(i)} This modification improves overall classification accuracy by moving the decision threshold to counter the bias toward majority classes, especially when combined with oversampled or baseline models, though it leaves multi-class ROC AUC unchanged because positive scaling preserves class rankings.

  5. Knowl 5 — Optimal Sampling Ratios for Oversampling and Undersampling in CNNs

    empirical result

    When varying the target balance ratio during resampling in CNN training:

    • Oversampling: Full oversampling—replicating minority instances until all classes match the sample count of the largest majority class (imbalance ratio ρ=1\rho = 1)—consistently yields the highest multi-class ROC AUC. Any reduction of imbalance improves performance over baseline training, and performance monotonically increases as imbalance is eliminated completely.
    • Undersampling: The optimal level of undersampling depends on the initial proportion of minority classes (μ\mu). When the number of minority classes is moderate, intermediate levels of undersampling (partially reducing majority class counts rather than equalizing down to the smallest minority class size) often outperform both full undersampling and the baseline. While there exists a partial undersampling ratio that matches oversampling performance, this optimal ratio cannot be determined a priori without empirical search.

  6. Knowl 6 — Scaling of Class Imbalance Sensitivity with Task and Dataset Complexity

    empirical result

    The detrimental impact of class imbalance on CNN classification performance scales sharply with the complexity of the visual classification task. On MNIST (a simple 10-class digit dataset trained with LeNet-5), high multi-class ROC AUC is maintained even under severe step and linear imbalance ratios (e.g., ρ=1000\rho = 1000 to 50005000). On CIFAR-10 (natural images trained with All-CNN), an equivalent relative drop in ROC AUC occurs at imbalance ratios roughly two orders of magnitude lower (around ρ=10\rho = 10 to 5050). On ImageNet (ILSVRC-2012, 1000 classes trained with ResNet-10), multi-class ROC AUC drops from 99.50%99.50\% on balanced data down to ∼88.2%\sim 88.2\% to 90.7%90.7\% under severe step imbalance (minority fraction μ=0.9\mu=0.9, imbalance ratio ρ=100\rho=100), demonstrating that deeper architectures and higher-dimensional natural image tasks are substantially more vulnerable to class imbalance.

  7. Knowl 7 — Performance of Two-Phase Training Compared to Standard Resampling

    empirical result

    Two-phase training—pre-training the entire convolutional neural network on a resampled (oversampled or undersampled) balanced dataset, followed by fine-tuning only the final classification layers on the original imbalanced training set with a reduced learning rate (scaled by 10−110^{-1})—consistently achieves multi-class ROC AUC intermediate between the unadjusted baseline model and the corresponding pure resampling method. Because standard oversampling almost universally outperforms baseline training, fine-tuning the output layer on the imbalanced dataset in the second phase degrades the performance of the oversampled model. Consequently, two-phase training offers no performance advantage over single-phase random minority oversampling.

  8. Knowl 8 — Performance Comparison of Imbalance Strategies on ImageNet (ILSVRC-2012)

    data/table

    The classification performance of a ResNet-10 architecture trained on artificially step-imbalanced subsets of ImageNet (ILSVRC-2012, 1,000 classes) was evaluated across baseline training, random minority oversampling, and random majority undersampling. The metric reported is multi-class ROC AUC (percentage). Three random runs were conducted for the most severe imbalance scenario (minority fraction μ=0.9\mu = 0.9, imbalance ratio ρ=100\rho = 100) to estimate variance:

    Method μ=0.1,ρ=10\mu = 0.1, \rho = 10 μ=0.8,ρ=50\mu = 0.8, \rho = 50 μ=0.9,ρ=100\mu = 0.9, \rho = 100 (3 runs)
    Baseline 99.41 96.31 90.74 90.46 90.05
    Oversampling 99.35 95.06 88.38 88.39 88.17
    Undersampling 96.85 94.98 88.35 84.08 83.74

    The results demonstrate that class imbalance degrades ImageNet discrimination performance from 99.50%99.50\% (balanced) down to ∼88−90%\sim 88 - 90\%. Oversampling consistently outperforms undersampling across all settings. At extreme imbalance (μ=0.9,ρ=100\mu = 0.9, \rho = 100, where the dataset contains only ∼10%\sim 10\% of original samples), baseline training yielded slightly higher multi-class ROC AUC than oversampling, though run-to-run variation (up to 4 AUC points in undersampling) and fixed hyperparameter confounding indicate that sampling methods on extreme ImageNet imbalance require careful hyperparameter tuning.

  9. Knowl 9 — Disentangling Class Imbalance Detriment from Sample Size Reduction

    empirical result

    The performance degradation observed in class-imbalanced CNNs is driven by the skewed class distribution rather than merely the reduction in total training samples:

    1. Oversampling vs. Baseline: Random minority oversampling uses the exact same set of unique training images as the baseline, yet altering the sampling frequency to equalize class appearance substantially improves multi-class ROC AUC.
    2. Step Imbalance vs. Linear Imbalance: For equal total training set sizes (comparing step imbalance with minority class fraction μ=0.5\mu = 0.5 against linear imbalance at the same imbalance ratio ρ\rho), the performance drop is significantly more severe for step imbalance than for linear imbalance.
    3. Undersampling Parity: In scenarios with high minority proportions (μ≥0.8\mu \ge 0.8), random undersampling (which discards vast amounts of training data) matches or outperforms the unadjusted baseline trained on substantially more total examples.

  10. Knowl 10 — Experimental Benchmarking Framework for CNN Class Imbalance

    experimental setup

    A systematic empirical framework evaluates seven imbalance mitigation strategies across three image classification benchmarks with fixed iteration counts:

    • Datasets and Models:
      • MNIST: 28×2828 \times 28 grayscale digits, 10 classes, LeNet-5 (2 conv layers, 2 max pooling, 2 fc layers). Trained with SGD (momentum 0.90.9, initial learning rate η0=0.01\eta_0 = 0.01 decaying via ηt=η0(1+0.0001t)−0.75\eta_t = \eta_0(1 + 0.0001 t)^{-0.75}, batch size 64, weight decay 0.00050.0005, 10,000 iterations). 50 randomized repeats per parameter setting.
      • CIFAR-10: 32×3232 \times 32 RGB images, 10 classes, All-CNN (9 conv layers with strided convolutions and average pooling). Trained with SGD (momentum 0.90.9, base learning rate searched in {0.05,0.005,0.0005,0.00005}\{0.05, 0.005, 0.0005, 0.00005\} with 0.1×0.1\times step drops at 40k, 50k, 60k iterations, batch size 256, weight decay 0.0010.001, 70,000 iterations).
      • ImageNet (ILSVRC-2012): 1,000 classes, ResNet-10 (4 residual blocks, 9 conv layers, 1 fc layer). Trained with SGD (momentum 0.90.9, linear learning rate decay from η0=0.1\eta_0 = 0.1 to 00, batch size 256, weight decay 0.00010.0001, 320,000 iterations).
    • Evaluation Metric: Multi-class ROC AUC, computed as the macro-average of one-vs-all ROC AUCs across all classes on a balanced test set, supplemented by overall classification accuracy.

Coverage note — None omitted; all core contributions including problem formulations, mitigation strategies, benchmark setups, and empirical findings are fully covered.

References

  1. 1.Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, and Gang Wang. Recent advances in convolutional neural networks. arXiv preprint arXiv:1512.07108, 2015.
  2. 2.Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer, 2014.
  3. 3.Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989.
  4. 4.Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  5. 5.Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  6. 6.Grant Van Horn, Oisin Mac Aodha, Yang Song, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist challenge 2017 dataset. arXiv preprint arXiv:1707.06642, 2017.
  7. 7.Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pages 3485–3492. IEEE, 2010.
  8. 8.Brian Alan Johnson, Ryutaro Tateishi, and Nguyen Thanh Hoan. A hybrid pansharpening approach and multiscale object-based image analysis for mapping diseased pine and oak trees. International journal of remote sensing, 34(20):6969–6982, 2013.
  9. 9.Miroslav Kubat, Robert C Holte, and Stan Matwin. Machine learning for the detection of oil spills in satellite radar images. Machine learning, 30(2-3):195–215, 1998.
  10. 10.Oscar Beijbom, Peter J Edmunds, David I Kline, B Greg Mitchell, and David Kriegman. Automated annotation of coral reef survey images. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 1170–1177. IEEE, 2012.
  11. 11.Jerzy W Grzymala-Busse, Linda K Goodwin, Witold J Grzymala-Busse, and Xinqun Zheng. An approach to imbalanced data sets based on changing rule strength. In Rough-Neural Computing, pages 543–553. Springer, 2004.
  12. 12.Brian Mac Namee, Padraig Cunningham, Stephen Byrne, and Owen I Corrigan. The problem of bias in training data in regression problems in medical decision support. Artificial intelligence in medicine, 24(1):51–70, 2002.
  13. 13.Philip K Chan and Salvatore J Stolfo. Toward scalable learning with non-uniform class and cost distributions: A case study in credit card fraud detection. In KDD, volume 1998, pages 164–168, 1998.
  14. 14.Predrag Radivojac, Nitesh V Chawla, A Keith Dunker, and Zoran Obradovic. Classification and knowledge discovery in protein databases. Journal of Biomedical Informatics, 37(4): 224–239, 2004.
  15. 15.Claire Cardie and Nicholas Howe. Improving minority class prediction using case-specific feature weights. In ICML, pages 57–65, 1997.
  16. 16.Guo Haixiang, Li Yijing, Jennifer Shang, Gu Mingyun, Huang Yuanyue, and Gong Bing. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 2016.
  17. 17.Nathalie Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intelligent data analysis, 6(5):429–449, 2002.
  18. 18.Maciej A Mazurowski, Piotr A Habas, Jacek M Zurada, Joseph Y Lo, Jay A Baker, and Georgia D Tourassi. Training neural network classifiers for medical decision making: The effects of imbalanced datasets on classification performance. Neural networks, 21(2):427–436, 2008.
  19. 19.Nitesh V Chawla. Data mining for imbalanced datasets: An overview. In Data mining and knowledge discovery handbook, pages 853–867. Springer, 2005.
  20. 20.Marcus A Maloof. Learning when data sets are imbalanced and when costs are unequal and unknown. In ICML-2003 workshop on learning from imbalanced data sets II, 2003.
  21. 21.Charles X Ling and Chenghui Li. Data mining for direct marketing: Problems and solutions. In KDD, volume 98, pages 73–79, 1998.
  22. 22.Zhi-Hua Zhou and Xu-Ying Liu. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on Knowledge and Data Engineering, 18(1):63–77, 2006.
  23. 23.Steve Lawrence, Ian Burns, Andrew Back, Ah Chung Tsoi, and C Lee Giles. Neural network classification and prior class probabilities. In Neural networks: tricks of the trade, pages 299–313. Springer, 1998.
  24. 24.Salman H Khan, Mohammed Bennamoun, Ferdous Sohel, and Roberto Togneri. Cost sensitive learning of deep feature representations from imbalanced data. arXiv preprint arXiv:1508.03422, 2015.
  25. 25.Vidwath Raj, Sven Magg, and Stefan Wermter. Towards effective classification of imbalanced data with convolutional neural networks. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pages 150–162. Springer, 2016.
  26. 26.Yu-An Chung, Hsuan-Tien Lin, and Shao-Wen Yang. Cost-aware pre-training for multiclass cost-sensitive deep learning. arXiv preprint arXiv:1511.09337, 2015.
  27. 27.Shoujin Wang, Wei Liu, Jia Wu, Longbing Cao, Qinxue Meng, and Paul J Kennedy. Training deep neural networks on imbalanced data sets. In Neural Networks (IJCNN), 2016 International Joint Conference on, pages 4368–4374. IEEE, 2016.
  28. 28.Mohammad Havaei, Axel Davy, David Warde-Farley, Antoine Biard, Aaron Courville, Yoshua Bengio, Chris Pal, Pierre-Marc Jodoin, and Hugo Larochelle. Brain tumor segmentation with deep neural networks. Medical image analysis, 35:18–31, 2017.
  29. 29.Haibo He and Edwardo A Garcia. Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9):1263–1284, 2009.
  30. 30.Gil Levi and Tal Hassner. Age and gender classification using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 34–42, 2015.
  31. 31.Andrew Janowczyk and Anant Madabhushi. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. Journal of pathology informatics, 7, 2016.
  32. 32.Nicolas Jaccard, Thomas W Rogers, Edward J Morton, and Lewis D Griffin. Detection of concealed cars in complex cargo x-ray imagery using deep learning. Journal of X-Ray Science and Technology, pages 1–17, 2016.
  33. 33.Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16: 321–357, 2002.
  34. 34.Kung-Jeng Wang, Bunjira Makond, Kun-Huang Chen, and Kung-Min Wang. A hybrid classifier combining smote with pso to estimate 5-year survivability of breast cancer patients. Applied Soft Computing, 20:15–24, 2014.
  35. 35.Hui Han, Wen-Yuan Wang, and Bing-Huan Mao. Borderline-smote: a new over-sampling method in imbalanced data sets learning. Advances in intelligent computing, pages 878–887, 2005.
  36. 36.Taeho Jo and Nathalie Japkowicz. Class imbalances versus small disjuncts. ACM Sigkdd Explorations Newsletter, 6(1):40–49, 2004.
  37. 37.Hongyu Guo and Herna L Viktor. Learning from imbalanced data sets with boosting and data generation: the databoost-im approach. ACM Sigkdd Explorations Newsletter, 6(1): 30–39, 2004.
  38. 38.Li Shen, Zhouchen Lin, and Qingming Huang. Relay backpropagation for effective learning of deep convolutional neural networks. In European Conference on Computer Vision, pages 467–482. Springer, 2016.
  39. 39.Chris Drummond, Robert C Holte, et al. C4.5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling. In Workshop on learning from imbalanced datasets II, volume 11, pages 1–8, 2003.
  40. 40.Miroslav Kubat, Stan Matwin, et al. Addressing the curse of imbalanced training sets: one-sided selection. In ICML, volume 97, pages 179–186. Nashville, USA, 1997.
  41. 41.Jack Koplowitz and Thomas A Brown. On the relation of performance to editing in nearest neighbor rules. Pattern Recognition, 13(3):251–255, 1981.
  42. 42.Ricardo Barandela, E Rangel, Jose Salvador Sanchez, and Francesc J Ferri. Restricted decontamination for the imbalanced training sample problem. In Iberoamerican Congress on Pattern Recognition, pages 424–431. Springer, 2003.
  43. 43.Michael D Richard and Richard P Lippmann. Neural network classifiers estimate bayesian a posteriori probabilities. Neural computation, 3(4):461–483, 1991.
  44. 44.Charles Elkan. The foundations of cost-sensitive learning. In International joint conference on artificial intelligence, volume 17, pages 973–978. Lawrence Erlbaum Associates Ltd, 2001.
  45. 45.Matjaz Kukar, Igor Kononenko, et al. Cost-sensitive learning with neural networks. In ECAI, pages 445–449, 1998.
  46. 46.Nathalie Japkowicz, Catherine Myers, Mark Gluck, et al. A novelty detection approach to classification. In IJCAI, volume 1, pages 518–523, 1995.
  47. 47.Nathalie Japkowicz, Stephen Jose Hanson, and Mark A Gluck. Nonlinear autoassociation is not equivalent to pca. Neural computation, 12(3):531–545, 2000.
  48. 48.Hoon Sohn, Keith Worden, and Charles R Farrar. Novelty detection using auto-associative neural network. In Symposium on Identification of Mechanical Systems: international mechanical engineering congress and exposition, New York, NY, pages 573–580, 2001.
  49. 49.Hyoung-joo Lee and Sungzoon Cho. The novelty detection approach for different degrees of class imbalance. In Neural Information Processing, pages 21–30. Springer, 2006.
  50. 50.Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2):539–550, 2009.
  51. 51.Nitesh V Chawla, Aleksandar Lazarevic, Lawrence O Hall, and Kevin W Bowyer. Smote-boost: Improving prediction of the minority class in boosting. In European Conference on Principles of Data Mining and Knowledge Discovery, pages 107–119. Springer, 2003.
  52. 52.Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
  53. 53.Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
  54. 54.Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhi-heng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015.
  55. 55.Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Gir-shick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pages 675–678. ACM, 2014.
  56. 56.Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 12(1):145–151, 1999.
  57. 57.Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Aistats, volume 9, pages 249–256, 2010.
  58. 58.Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron C Courville, and Yoshua Bengio. Maxout networks. ICML (3), 28:1319–1327, 2013.
  59. 59.Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.
  60. 60.Marcel Simon, Erik Rodner, and Joachim Denzler. Imagenet pre-trained models with batch normalization. arXiv preprint arXiv:1612.01452, 2016.
  61. 61.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  62. 62.Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
  63. 63.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  64. 64.Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7):1145–1159, 1997.
  65. 65.Fabian Pedregosa, Gael Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. Journal of machine learning research, 12 (Oct):2825–2830, 2011.
  66. 66.Charles X Ling, Jin Huang, and Harry Zhang. Auc: a statistically consistent and more discriminating measure than accuracy. In IJCAI, volume 3, pages 519–524, 2003.
  67. 67.Foster Provost and Pedro Domingos. Tree induction for probability-based ranking. Machine learning, 52(3):199–215, 2003.
  68. 68.Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. (Master’s thesis) Royal Institute of Technology (KTH), 2017. Retrieved from http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-219872

Citation

MLA
Buda, M., et al. “A Systematic Study of the Class Imbalance Problem in Convolutional Neural Networks”. Neural Networks, vol. 106, 2018, pp. 249–59, https://doi.org/10.1016/j.neunet.2018.07.011.
APA
Buda, M., Maki, A., & Mazurowski, M. A. (2018). A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106, 249–259. https://doi.org/10.1016/j.neunet.2018.07.011
Chicago
Buda, M., A. Maki, and M. A. Mazurowski. 2018. “A Systematic Study of the Class Imbalance Problem in Convolutional Neural Networks”. Neural Networks 106: 249–59. https://doi.org/10.1016/j.neunet.2018.07.011.
Harvard
Buda, M., Maki, A. and Mazurowski, M.A. (2018) “A systematic study of the class imbalance problem in convolutional neural networks”, Neural Networks, 106, pp. 249–259. Available at: https://doi.org/10.1016/j.neunet.2018.07.011.
Vancouver
1. Buda M, Maki A, Mazurowski MA (2018) A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks 106:249–259

BibTeX

@article{Buda_2018, title={A systematic study of the class imbalance problem in convolutional neural networks}, volume={106}, ISSN={0893-6080}, url={http://dx.doi.org/10.1016/j.neunet.2018.07.011}, DOI={10.1016/j.neunet.2018.07.011}, journal={Neural Networks}, publisher={Elsevier BV}, author={Buda, Mateusz and Maki, Atsuto and Mazurowski, Maciej A.}, year={2018}, month=Oct, pages={249–259} }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF