Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding

Lan-Zhe GuoYufeng Li

article2022ICML172 citations

Proposes a class-dependent adaptive thresholding method with an efficient closed-form solution to improve pseudo-label selection and classification accuracy on minority classes in imbalanced semi-supervised learning.

Listen

Practical machine learning applications frequently face severe class imbalance alongside high data-labeling costs, leading to heavy reliance on semi-supervised learning techniques that use large volumes of unlabeled data. However, existing semi-supervised methods usually assume balanced categories and apply a single, fixed confidence threshold to assign pseudo-labels to unlabeled samples. This fixed-threshold approach severely biases model predictions toward majority categories, disproportionately discarding useful minority-class samples and causing substantial performance degradation on rare classes in real-world deployments.

The article evaluates and demonstrates a novel framework called Adaptive Thresholding (Adsh), which adjusts pseudo-label selection thresholds individually for each class during model training. The main objective is to establish an efficient, distribution-aware learning method that simultaneously minimizes prediction error and balances pseudo-label generation across imbalanced classes, without requiring prior knowledge of unlabeled data distributions.

To demonstrate this method, the authors developed a mathematical formulation that integrates class-specific selection biases into the learning objective, yielding an exact, efficient solution for setting adaptive thresholds. The framework was comprehensively evaluated using a standard deep neural network architecture across long-tailed variants of standard image benchmarks (CIFAR-10, SVHN, and STL-10). The experiments encompassed over twenty distinct imbalance ratios, varying volumes of labeled samples, and challenging scenarios where the class distributions of labeled and unlabeled sets differed significantly.

The primary findings show that Adsh consistently outperforms existing state-of-the-art semi-supervised and imbalanced-learning techniques across all evaluated settings. In heavily imbalanced scenarios on CIFAR-10, Adsh improved classification accuracy by up to 4 to 5 percentage points over leading baselines like FixMatch, DARP, and CReST. On SVHN and STL-10 datasets, Adsh achieved top accuracies of 92.13% and 79.25% respectively, demonstrating strong adaptability even when unlabeled class distributions were unknown. Furthermore, error analyses revealed that Adsh produced substantially less biased confusion matrices, and the framework readily combined with downstream re-balancing techniques to boost overall accuracy up to 86.21%.

These findings indicate that adapting confidence thresholds by class significantly enhances the robustness and performance of automated prediction systems in skewed environments, reducing operational risks associated with minority-class misclassification. Practitioners can implement Adsh with minimal computational overhead, as it operates via closed-form updates and avoids complex hyperparameter tuning. Organizations should consider Adsh as a drop-in enhancement for semi-supervised pipelines facing rare-event or class-skew challenges, starting with the recommended baseline configuration and fine-tuning as needed.

While empirical results strongly support the framework's practical efficacy across diverse vision datasets, the authors note that formal theoretical convergence guarantees remain an area for future research. Stakeholders should maintain high confidence in Adsh's empirical gains while piloting the approach across domain-specific data types to validate generalization beyond standard vision benchmarks.

Nanjing University closed-form solution semi-supervised learning supervised machine learning Class-imbalanced semi-supervised learning Adaptive thresholding Pseudo-labeled examples

guolz-ml/Class-Imbalanced-SSL Guo et al (2022).pdf

Paper: FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence, Kihyuk Sohn et al. (2020). FixMatch establishes the confidence-thresholded pseudo-labeling objective that this paper adapts to class imbalance.
Paper: FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling, Bowen Zhang et al. (2021). FlexMatch develops class-specific adaptive confidence thresholds, providing the key prior SSL mechanism that this paper redirects toward balanced pseudo-label selection.
Paper: A survey on semi-supervised learning, Jesper E. van Engelen et al. (2019). This survey organizes pseudo-labeling and other SSL approaches, helping situate the paper’s threshold-based method within the field’s established frameworks.

Paper: Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data, Yuhao Chen et al. (2023). FullMatch extends confidence-filtered SSL by extracting supervisory signals from low-confidence examples as well as selected pseudo-labels, broadening the thresholding problem this paper addresses.

Cover for Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding

Abstract

Semi-supervised learning (SSL) has proven to be successful in overcoming labeling difficulties by leveraging unlabeled data. Previous SSL algorithms typically assume a balanced class distribution. However, real-world datasets are usually class-imbalanced, causing the performance of existing SSL algorithms to be seriously decreased. One essential reason is that pseudo-labels for unlabeled data are selected based on a fixed confidence threshold, resulting in low performance on minority classes. In this paper, we develop a simple yet effective framework, which only involves adaptive thresholding for different classes in SSL algorithms, and achieves remarkable performance improvement on more than twenty imbalance ratios. Specifically, we explicitly optimize the number of pseudo-labels for each class in the SSL objective, so as to simultaneously obtain adaptive thresholds and minimize empirical risk. Moreover, the determination of the adaptive threshold can be efficiently obtained by a closed-form solution. Extensive experimental results demonstrate the effectiveness of our proposed algorithms.

Topics

closed-form solution semi-supervised learning supervised machine learning thresholding unsupervised learning adaptive algorithms

Keywords

Class-imbalanced semi-supervised learning Adaptive thresholding Pseudo-labeled examples Consistency regularization CIFAR-10 dataset

1. Introduction
2. Related Works
3. Preliminary and Background
3.1. Problem Setting and Notations
3.2. FixMatch: An SSL algorithm with Fixed Thresholding
4. Adsh: An SSL Algorithm with Adaptive Thresholding
5. Experiments
5.1. Experimental setup
5.2. Empirical Results
5.3. Detailed Analyses
6. Conclusions
Acknowledgements
References
A. Theorem Proof
B. Implementation Details
C. Combination of Class-Imbalanced Learning and SSL

Knowls

Knowl 1 — Adsh Optimization Objective for Class-Imbalanced Semi-Supervised Learning
model/method
In class-imbalanced semi-supervised learning, fixed confidence thresholding causes minority class pseudo-labels to be systematically filtered out due to model prediction bias toward majority classes. The Adaptive Thresholding (Adsh) framework addresses this by explicitly integrating class-dependent selection bias variables into the empirical risk minimization objective.

Let $\mathcal{D}^l = \{(\mathbf{x}_i^l, \mathbf{y}_i^l)\}_{i=1}^N$ be the labeled dataset with $N$ examples and one-hot labels $\mathbf{y}_i \in \{0, 1\}^K$ , and let $\mathcal{D}^u = \{\mathbf{x}_i^u\}_{i=1}^M$ be the unlabeled dataset with $M$ examples. Let $f(y=k|\mathbf{x}; \theta) \in [0, 1]$ denote the model's predicted probability for class $k$ parameterized by $\theta$ , and let $\alpha(\cdot)$ denote a weak data augmentation function. Let $\hat{\mathbf{y}}_i = [\hat{y}_{i,1}, \dots, \hat{y}_{i,K}]^\top \in \{0, 1\}^K$ represent the discrete pseudo-label vector for unlabeled instance $\mathbf{x}_i^u$ , where $\mathbf{0} \le \mathbf{1}^\top \hat{\mathbf{y}}_i \le 1$ allows an example to remain unselected (assigned a zero vector). To balance pseudo-label selection across classes, a class-wise bias vector $\mathbf{s} = [s_1, \dots, s_K]^\top \in \mathbb{R}_{>0}^K$ is introduced, where larger $s_k$ values promote the selection of more pseudo-labeled instances for class $k$ .

The joint optimization problem over $\hat{\mathbf{y}}, \mathbf{s}$ , and $\theta$ is formulated as:

$\min_{\hat{\mathbf{y}}, \mathbf{s}, \theta} \frac{1}{N} \sum_{i=1}^N \sum_{k=1}^K -y_{i,k} \log f(y=k|\alpha(\mathbf{x}_i^l); \theta) + \frac{1}{M} \sum_{i=1}^M \sum_{k=1}^K \left[ -\hat{y}_{i,k} \log f(y=k|\alpha(\mathbf{x}_i^u); \theta) - s_k \hat{y}_{i,k} \right]$

$\text{subject to } \hat{\mathbf{y}}_i \in \{0, 1\}^K, \quad 0 \le \mathbf{1}^\top \hat{\mathbf{y}}_i \le 1, \quad s_k > 0, \; \forall 1 \le k \le K$
Knowl 2 — Closed-Form Solution for Pseudo-Labels in Adsh
theoretical result
Given a fixed model parameter $\theta$ and fixed class bias parameters $s_k > 0$ for each class $k \in \{1, \dots, K\}$ , the optimal pseudo-label $\hat{\mathbf{y}}_i = [\hat{y}_{i,1}, \dots, \hat{y}_{i,K}]^\top$ for an unlabeled sample $\mathbf{x}_i^u$ in the Adsh objective has the exact closed-form solution:

$\hat{y}_{i,k} = \begin{cases} 1, & \text{if } k = \arg\max_c \frac{f(y=c|\alpha(\mathbf{x}_i^u); \theta)}{\exp(-s_c)} \quad \text{and} \quad \frac{f(y=k|\alpha(\mathbf{x}_i^u); \theta)}{\exp(-s_k)} \ge 1, \\ 0, & \text{otherwise.} \end{cases}$

This indicates that an unlabeled instance $\mathbf{x}_i^u$ is assigned pseudo-label $k$ if and only if class $k$ achieves the maximal scaled prediction and the model confidence satisfies $f(y=k|\alpha(\mathbf{x}_i^u); \theta) \ge \exp(-s_k)$ , establishing $\tau_k = \exp(-s_k)$ as the class-dependent adaptive confidence threshold.
Knowl 3 — Class Prediction Invariance Under Adaptive Scaling
theoretical result
Let $f(y=k|\alpha(\mathbf{x}_i^u); \theta)$ be the model prediction on weak augmentation $\alpha(\mathbf{x}_i^u)$ and $s_k > 0$ be the class bias parameters. If for all classes $k$ and $k'$ satisfying $f(y=k|\alpha(\mathbf{x}_i^u); \theta) > f(y=k'|\alpha(\mathbf{x}_i^u); \theta)$ , the condition

$\exp(s_k - s_{k'}) > \frac{f(y=k'|\alpha(\mathbf{x}_i^u); \theta)}{f(y=k|\alpha(\mathbf{x}_i^u); \theta)}$

holds, then the class achieving the maximal ratio equals the class with the maximal unscaled model probability:

$\arg\max_k \frac{f(y=k|\alpha(\mathbf{x}_i^u); \theta)}{\exp(-s_k)} = \arg\max_k f(y=k|\alpha(\mathbf{x}_i^u); \theta)$

Because deep neural networks trained with semi-supervised learning are typically over-confident on selected examples, the probability ratio $f(y=k'|\alpha(\mathbf{x}_i^u); \theta)/f(y=k|\alpha(\mathbf{x}_i^u); \theta)$ for non-maximal classes is near zero, making this condition easily satisfied in practice. Consequently, pseudo-label selection simplifies to selecting samples predicted as $\hat{y}_i^u = \arg\max_k f(y=k|\alpha(\mathbf{x}_i^u); \theta)$ whose maximum confidence satisfies $\max_k f(y=k|\alpha(\mathbf{x}_i^u); \theta) \ge \exp(-s_{\hat{y}_i^u})$ .
Knowl 4 — Class Bias Computation Under Known and Unknown Unlabeled Distributions
algorithm
The class bias vector $\mathbf{s} = [s_1, \dots, s_K]$ determines the adaptive threshold $\tau_k = \exp(-s_k)$ for each class $k$ .

When the ground-truth unlabeled class distribution ratio $\gamma_k = (\sum_{i=1}^M y_{i,1}^*) / (\sum_{i=1}^M y_{i,k}^*)$ is known, given a user-defined threshold $\tau_1$ for the majority class (e.g., $s_1 = -\log(0.95)$ ), each $s_k$ ( $2 \le k \le K$ ) is chosen to satisfy:

$\sum_{i=1}^M \mathbb{I}(f(y=k|\alpha(\mathbf{x}_i^u); \theta) \ge \exp(-s_k)) = \frac{1}{\gamma_k} \sum_{i=1}^M \mathbb{I}(f(y=1|\alpha(\mathbf{x}_i^u); \theta) \ge \exp(-s_1))$

When the unlabeled class distribution is unknown, $s_k$ is computed by matching within-class confidence percentiles across classes as detailed below:
Input: Model parameter $\theta$ , unlabeled data $\mathcal{D}^u = \{\mathbf{x}_i^u\}_{i=1}^M$ , number of classes $K$ , user-defined majority class threshold $\tau_1$
Output: Class bias vector $\mathbf{s} \in \mathbb{R}^K$
Initialize empty lists $C_1, C_2, \dots, C_K$
for each $\mathbf{x}^u \in \mathcal{D}^u$ do
     $\mathbf{q} = f(y|\alpha(\mathbf{x}^u); \theta)$
     $\hat{y} = \arg\max(\mathbf{q})$
    Append $\max(\mathbf{q})$ to list $C_{\hat{y}}$
end for
Sort list $C_k$ in descending order for all $k \in \{1, \dots, K\}$
len_selected = 0
for $j = 1$ to $\text{length}(C_1)$ do
    if $C_1[j] < \tau_1$ then
        break
    end if
    len_selected = $j$
end for
$\rho = \text{len_selected} / \text{length}(C_1)$
for $k = 1$ to $K$ do
    idx = $\lfloor \text{length}(C_k) \times \rho \rfloor$
     $s_k = -\log(C_k[\max(1, \text{idx})])$
end for
return $\mathbf{s}$
Knowl 5 — Adsh Training Algorithm
algorithm
The Adsh algorithm optimizes network parameters $\theta$ and updates class-dependent thresholds throughout training.
Input: Labeled data $\mathcal{D}^l$ , unlabeled data $\mathcal{D}^u$ , classes $K$ , epochs $E$ , iterations $T$ , unlabeled loss weight $\lambda_u$ , unlabeled ratio $\mu$ , initial class bias $\mathbf{s} \in \mathbb{R}^K$ , initial parameters $\theta_0$ , majority threshold $\tau_1$
Output: Trained model parameters $\theta_T$
$t = 0$
for $e = 1$ to $E$ do
    for iter = 1 to $\lfloor T/E \rfloor$ do
        Sample labeled batch $\{(\mathbf{x}_b^l, \mathbf{y}_b^l)\}_{b=1}^B$ from $\mathcal{D}^l$
        Sample unlabeled batch $\{\mathbf{x}_b^u\}_{b=1}^{\mu B}$ from $\mathcal{D}^u$

         $\mathcal{L}_s = \frac{1}{B} \sum_{b=1}^B H(\mathbf{y}_b^l, f(y|\alpha(\mathbf{x}_b^l); \theta_t))$

        for $b = 1$ to $\mu B$ do
             $\mathbf{q}_b = f(y|\alpha(\mathbf{x}_b^u); \theta_t)$
             $\hat{y}_b^u = \arg\max(\mathbf{q}_b)$
             $H_b = H(\hat{y}_b^u, f(y|A(\mathbf{x}_b^u); \theta_t))$
        end for

        for $k = 1$ to $K$ do
             $\tau_k = \exp(-s_k)$
        end for

         $\mathcal{L}_u = \frac{1}{\mu B} \sum_{b=1}^{\mu B} \mathbb{I}(\max(\mathbf{q}_b) \ge \tau_{\hat{y}_b^u}) H_b$
         $\mathcal{L} = \mathcal{L}_s + \lambda_u \mathcal{L}_u$

         $\theta_{t+1} = \text{OptimizationStep}(\theta_t, \mathcal{L})$
         $t = t + 1$
    end for
    Update $\mathbf{s}$ using percentile threshold calculation on $\mathcal{D}^u$
end for
return $\theta_T$
Here, $\alpha(\cdot)$ and $A(\cdot)$ denote weak (e.g., standard crop/flip) and strong (e.g., RandAugment) data augmentations respectively, and $H(\cdot, \cdot)$ is the standard cross-entropy loss.

Knowl 6 — Classification Accuracy on Imbalanced CIFAR-10 with Matched Imbalance

data/table

Models were trained using Wide ResNet-28-2 on long-tailed CIFAR-10 where labeled and unlabeled data have identical imbalance ratios ( $\gamma = \gamma_l = \gamma_u \in \{50, 100, 150\}$ ), evaluating two dataset size configurations: $(N_1 = 1500, M_1 = 3000)$ and $(N_1 = 500, M_1 = 4000)$ . Test accuracy (%) is measured on a balanced test set over five runs (mean $\pm$ standard deviation).

	$N_1 = 1500, M_1 = 3000$			$N_1 = 500, M_1 = 4000$
Algorithm	$\gamma = 50$	$\gamma = 100$	$\gamma = 150$	$\gamma = 50$	$\gamma = 100$	$\gamma = 150$
Supervised	$65.23 \pm 0.05$	$58.94 \pm 0.13$	$55.63 \pm 0.38$	$51.31 \pm 0.34$	$45.82 \pm 0.41$	$40.90 \pm 0.39$
CBL	$65.52 \pm 0.31$	$58.52 \pm 0.45$	$52.36 \pm 0.58$	$51.94 \pm 0.71$	$46.22 \pm 0.92$	$41.58 \pm 1.24$
Re-Sampling	$64.53 \pm 0.39$	$56.34 \pm 0.42$	$53.21 \pm 0.51$	$51.96 \pm 0.65$	$48.13 \pm 1.25$	$40.26 \pm 1.88$
cRT	$67.82 \pm 0.14$	$63.43 \pm 0.45$	$59.56 \pm 0.44$	$56.28 \pm 1.45$	$48.11 \pm 0.79$	$45.02 \pm 1.08$
LDAM	$68.91 \pm 0.10$	$63.15 \pm 0.24$	$58.68 \pm 0.30$	$56.41 \pm 0.92$	$49.27 \pm 0.88$	$45.10 \pm 0.75$
Mean-Teacher	$68.84 \pm 0.82$	$61.33 \pm 0.28$	$54.79 \pm 0.31$	$56.34 \pm 1.68$	$48.55 \pm 0.77$	$45.32 \pm 1.20$
MixMatch	$73.59 \pm 0.46$	$65.03 \pm 0.26$	$62.71 \pm 0.29$	$65.32 \pm 1.20$	$56.41 \pm 1.96$	$52.38 \pm 1.88$
ReMixMatch	$78.96 \pm 0.29$	$72.88 \pm 0.12$	$68.61 \pm 0.40$	$76.83 \pm 0.98$	$70.12 \pm 1.23$	$59.58 \pm 1.30$
FixMatch	$79.10 \pm 0.14$	$71.50 \pm 0.31$	$68.47 \pm 0.15$	$77.34 \pm 0.96$	$68.45 \pm 0.94$	$60.10 \pm 0.82$
DARP	$81.60 \pm 0.31$	$75.23 \pm 0.14$	$69.31 \pm 0.26$	$76.72 \pm 0.46$	$69.41 \pm 0.50$	$61.23 \pm 0.31$
CReST	$82.03 \pm 0.26$	$75.08 \pm 0.41$	$69.84 \pm 0.39$	$76.18 \pm 0.36$	$69.50 \pm 0.70$	$60.81 \pm 0.55$
Adsh	$\mathbf{83.38 \pm 0.06}$	$\mathbf{76.52 \pm 0.35}$	$\mathbf{71.49 \pm 0.30}$	$\mathbf{79.27 \pm 0.38}$	$\mathbf{70.97 \pm 0.46}$	$\mathbf{62.04 \pm 0.51}$

Adsh consistently outperforms all supervised baselines, classic semi-supervised methods, and specialized class-imbalanced semi-supervised methods (DARP and CReST) across all imbalance ratios and labeled sample sizes.

Knowl 7 — Robustness of Adsh Under Mismatched Imbalance Ratios
empirical result
When evaluated on CIFAR-10 with mismatched imbalance ratios between labeled data ( $\gamma_l \in \{50, 100, 150\}$ ) and unlabeled data ( $\gamma_u \in \{1, 50, 100, 150\}$ ) under both $N_1 = 1500$ and $N_1 = 500$ settings (spanning 24 combinations):
1. Across all $\gamma_l$ and $\gamma_u$ configurations, Adsh consistently outperforms FixMatch and DARP. DARP exhibits severe performance drops when distributions are mismatched, sometimes performing worse than vanilla FixMatch due to reliance on erroneous distribution estimations.
2. When unlabeled data is perfectly balanced ( $\gamma_u = 1$ ) but labeled data is imbalanced ( $\gamma_l \in \{50, 100, 150\}$ ), all semi-supervised methods suffer an unexpected performance drop compared to moderately imbalanced unlabeled cases (e.g., $\gamma_u = 50$ ), demonstrating that extreme distribution mismatch between labeled and unlabeled sets impedes SSL gains.

Knowl 8 — Classification Accuracy on Imbalanced SVHN and STL-10

data/table

Classification accuracy (%) on imbalanced SVHN (with 20% labeled data and $\gamma_l = \gamma_u = 100$ ) and STL-10 (with $N_1 = 450$ , labeled imbalance ratio $\gamma_l \in \{10, 20\}$ , and $M = 100{,}000$ unlabeled samples having unknown class distribution $\gamma_u$ ). Evaluated on balanced test sets over five runs (mean $\pm$ std):

	SVHN	STL-10
Algorithm	$\gamma = 100$	$\gamma_l = 10$	$\gamma_l = 20$
ReMixMatch	$88.91 \pm 0.32$	$67.43 \pm 0.43$	$60.82 \pm 0.93$
FixMatch	$89.34 \pm 0.20$	$73.25 \pm 0.21$	$63.54 \pm 0.21$
DARP	$90.15 \pm 0.46$	$76.97 \pm 0.45$	$68.87 \pm 0.66$
CReST	$89.90 \pm 0.64$	$76.30 \pm 0.38$	$69.43 \pm 0.89$
Adsh	$\mathbf{92.13 \pm 0.39}$	$\mathbf{79.25 \pm 0.41}$	$\mathbf{71.03 \pm 0.20}$

Adsh achieves superior accuracy over both standard SSL methods and class-imbalanced SSL algorithms on both datasets, including on STL-10 where the true unlabeled distribution is unknown and mismatched with the labeled distribution.

Knowl 9 — Combining Adsh with Classifier Re-Training (cRT)

data/table

Adsh can be integrated with supervised class-rebalancing techniques such as classifier Re-Training (cRT), where the feature extractor is trained via SSL and the classification head is subsequently retrained with a class-balanced sampling objective. Classification accuracy (%) on CIFAR-10 with $N_1 = 1500, M_1 = 3000$ across $\gamma = \gamma_l = \gamma_u \in \{50, 100, 150\}$ :

Algorithm	$\gamma = 50$	$\gamma = 100$	$\gamma = 150$
Supervised	$65.23 \pm 0.05$	$58.94 \pm 0.13$	$55.63 \pm 0.38$
cRT	$67.82 \pm 0.14$	$63.43 \pm 0.45$	$59.56 \pm 0.44$
FixMatch	$79.10 \pm 0.14$	$71.50 \pm 0.31$	$68.47 \pm 0.15$
Adsh	$83.38 \pm 0.06$	$76.52 \pm 0.35$	$71.49 \pm 0.30$
FixMatch + cRT	$84.32 \pm 0.40$	$78.39 \pm 0.45$	$73.26 \pm 0.23$
Adsh + cRT	$\mathbf{86.21 \pm 0.24}$	$\mathbf{79.82 \pm 0.24}$	$\mathbf{75.48 \pm 0.31}$

Combining Adsh with cRT provides additive accuracy improvements of 2.83% to 3.99% over Adsh alone, outperforming FixMatch + cRT across all imbalance ratios.

Knowl 10 — Theoretical Convergence Limitation of Adsh
limitation
While the pseudo-label assignment in Adsh possesses a closed-form solution for fixed model parameters $\theta$ and class bias vector $\mathbf{s}$ , the alternating optimization framework lacks formal theoretical convergence guarantees for deep neural network models. Establishing rigorous convergence bounds comparing fixed thresholding and adaptive class-dependent thresholding in non-convex semi-supervised learning remains an open problem.

Coverage note — None was omitted; all key contributions including the formulation, closed-form theorem, adaptive bias algorithms, main benchmark tables, mismatched ratio experiments, combination with cRT, and stated theoretical limitations are fully covered.

References

1.Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems, pp. 5050–5060, 2019.
2.Berthelot, D., Carlini, N., Cubuk, E. D., Kurakin, A., Sohn, K., Zhang, H., and Raffel, C. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. In Proceedings of the 8th International Conference on Learning Representations, 2020.
3.Buda, M., Maki, A., and Mazurowski, M. A. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249–259, 2018.
4.Byrd, J. and Lipton, Z. What is the effect of importance weighting in deep learning? In Proceedings of the 36th International Conference on Machine Learning, pp. 872–881, 2019.
5.Cao, K., Wei, C., Gaidon, A., Aréchiga, N., and Ma, T. Learning imbalanced datasets with label-distribution-aware margin loss. In Advances in Neural Information Processing Systems, pp. 1565–1576, 2019.
6.Chapelle, O., Scholkopf, B., and Zien, A. Semi-supervised learning. MIT Press, 2006.
7.Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321–357, 2002.
8.Coates, A., Ng, A. Y., and Lee, H. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, pp. 215–223, 2011.
9.Cui, Y., Jia, M., Lin, T.-Y., Song, Y., and Belongie, S. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9268–9277, 2019.
10.Dong, Q., Gong, S., and Zhu, X. Imbalanced deep learning by minority class incremental rectification. IEEE Transaction on Pattern Analysis and Machine Intelligence, 41 (6):1367–1381, 2019.
11.Grandvalet, Y. and Bengio, Y. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems, pp. 529–536, 2005.
12.Guo, L.-Z., Zhang, Z.-Y., Jiang, Y., Li, Y.-F., and Zhou, Z.-H. Safe deep semi-supervised learning for unseen-class unlabeled data. In Proceedings of the 37th International Conference on Machine Learning, pp. 3897–3906, 2020a.
13.Guo, L. Z., Zhou, Z., and Li, Y. F. RECORD: resource constrained semi-supervised learning under distribution shift. In Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1636–1644, 2020b.
14.Guo, L. Z., Zhou, Z., Shao, J. J., Zhang, Q., Kuang, F., Li, G. L., Liu, Z. X., Wu, G., Ma, N., Li, Q., and Li, Y. F. Learning from imbalanced and incomplete supervision with its application to ride-sharing liability judgment. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 487–495, 2021.
15.Guo, L. Z., Zhou, Z., and Li, Y. F. Robust deep semi-supervised learning: A brief introduction. CoRR, abs/2202.05975, 2022.
16.Hariharan, B. and Girshick, R. Low-shot visual recognition by shrinking and hallucinating features. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3018–3027, 2017.
17.He, H. and Garcia, E. A. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9):1263–1284, 2009.
18.Hu, Z., Tan, B., Salakhutdinov, R., Mitchell, T. M., and Xing, E. P. Learning data manipulation for augmentation and weighting. In Advances in Neural Information Processing Systems, pp. 15738–15749, 2019.
19.Huang, C., Li, Y., Loy, C. C., and Tang, X. Deep imbalanced learning for face recognition and attribute prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(11):2781–2794, 2019.
20.Jeong, J., Lee, S., Kim, J., and Kwak, N. Consistency-based semi-supervised learning for object detection. In Advances in Neural Information Processing Systems, pp. 10759–10768, 2019.
21.Johnson, J. M. and Khoshgoftaar, T. M. Survey on deep learning with class imbalance. Journal of Big Data, 6(1):1–54, 2019.
22.Kang, B., Xie, S., Rohrbach, M., Yan, Z., Gordo, A., Feng, J., and Kalantidis, Y. Decoupling representation and classifier for long-tailed recognition. In Proceedings of the 8th International Conference on Learning Representations, 2020.
23.Khan, S., Hayat, M., Zamir, S. W., Shen, J., and Shao, L. Striking the right balance with uncertainty. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 103–112, 2019.
24.Khan, S. H., Hayat, M., Bennamoun, M., Sohel, F. A., and Togneri, R. Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Transactions on Neural Networks and Learning Systems, 29(8):3573–3587, 2017.
25.Kim, J., Hur, Y., Park, S., Yang, E., Hwang, S. J., and Shin, J. Distribution aligning refinery of pseudo-label for imbalanced semi-supervised learning. In Advances in Neural Information Processing Systems, pp. 14567–14579, 2020.
26.Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015.
27.Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. Technical Report, 2009.
28.Laine, S. and Aila, T. Temporal ensembling for semi-supervised learning. In Proceedings of the 5th International Conference on Learning Representations, 2017.
29.LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. Nature, 521(7553):436–444, 2015.
30.Lee, D.-H. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML Workshop on Challenges in Representation Learning, pp. 2–8, 2013.
31.Li, Y.-F., Guo, L.-Z., and Zhou, Z.-H. Towards safe weakly supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(1):334–346, 2019.
32.Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988, 2017.
33.Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., and Yu, S. X. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2537–2546, 2019.
34.Miyato, T., Dai, A. M., and Goodfellow, I. J. Adversarial training methods for semi-supervised text classification. In Proceedings of the 5th International Conference on Learning Representations, 2017.
35.Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 (8):1979–1993, 2018.
36.Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NeurIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
37.Oliver, A., Odena, A., Raffel, C. A., Cubuk, E. D., and Goodfellow, I. Realistic evaluation of deep semi-supervised learning algorithms. In Advances in Neural Information Processing Systems, pp. 3235–3246, 2018.
38.Ren, M., Zeng, W., Yang, B., and Urtasun, R. Learning to reweight examples for robust deep learning. In Proceedings of the 35th International Conference on Machine Learning, pp. 4331–4340, 2018.
39.Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C., Cubuk, E. D., Kurakin, A., and Li, C. L. Fix-match: Simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems, pp. 596–608, 2020.
40.Souly, N., Spampinato, C., and Shah, M. Semi supervised semantic segmentation using generative adversarial network. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5688–5696, 2017.
41.Tarvainen, A. and Valpola, H. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems, pp. 1195–1204, 2017.
42.Thulasidasan, S., Chennupati, G., Bilmes, J. A., Bhattacharya, T., and Michalak, S. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances in Neural Information Processing Systems, pp. 13888–13899, 2019.
43.Wang, Y.-X., Ramanan, D., and Hebert, M. Learning to model the tail. In Advances in Neural Information Processing Systems, pp. 7032–7042, 2017.
44.Wei, C., Sohn, K., Mellina, C., Yuille, A. L., and Yang, F. Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning. In Proceedings of the IEEE International Conference on Computer Vision, 2021.
45.Xie, Q., Dai, Z., Hovy, E. H., Luong, T., and Le, Q. Unsupervised data augmentation for consistency training. In Advances in Neural Information Processing Systems, pp. 6256–6268, 2020.
46.Xu, Y., Shang, L., Ye, J., Qian, Q., Li, Y.-F., Sun, B., Li, H., and Jin, R. Dash: Semi-supervised learning with dynamic thresholding. In Proceedings of the 38th International Conference on Machine Learning, pp. 11525–11536, 2021.
47.Yin, X., Yu, X., Sohn, K., Liu, X., and Chandraker, M. Feature transfer learning for face recognition with under-represented data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5704–5713, 2019.
48.Zagoruyko, S. and Komodakis, N. Wide residual networks. In Proceedings of the British Machine Vision Conference, 2016.
49.Zhou, Z., Guo, L. Z., Cheng, Z., Li, Y. F., and Pu, S. STEP: out-of-distribution detection in the presence of limited in-distribution labeled data. In Advances in Neural Information Processing Systems, pp. 29168–29180, 2021.
50.Zhou, Z.-H. A brief introduction to weakly supervised learning. National Science Review, 5(1):44–53, 2017.
51.Zou, Y., Yu, Z., Kumar, B. V. K. V., and Wang, J. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European Conference on Computer Vision, pp. 297–313, 2018.
52.Zou, Y., Yu, Z., Liu, X., Kumar, B. V. K. V., and Wang, J. Confidence regularized self-training. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5981–5990, 2019.

Citation

MLA

Guo, L.-Z., and Y.-F. Li. “Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding”. International Conference on Machine Learning, vol. 162, 2022, pp. 8082–94, https://proceedings.mlr.press/v162/guo22e.html.

APA

Guo, L.-Z., & Li, Y.-F. (2022). Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding. International Conference on Machine Learning, 162, 8082–8094. https://proceedings.mlr.press/v162/guo22e.html

Chicago

Guo, L.-Z., and Y.-F. Li. 2022. “Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding”. International Conference on Machine Learning 162: 8082–94. https://proceedings.mlr.press/v162/guo22e.html.

Harvard

Guo, L.-Z. and Li, Y.-F. (2022) “Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding”, International Conference on Machine Learning. PMLR, pp. 8082–8094. Available at: https://proceedings.mlr.press/v162/guo22e.html.

Vancouver

1. Guo L-Z, Li Y-F (2022) Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding. In: International Conference on Machine Learning. PMLR, pp 8082–8094

BibTeX

@InProceedings{pmlr-v162-guo22e,
  title = 	 {Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding},
  author =       {Guo, Lan-Zhe and Li, Yu-Feng},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {8082--8094},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/guo22e/guo22e.pdf},
  url = 	 {https://proceedings.mlr.press/v162/guo22e.html},
  abstract = 	 {Semi-supervised learning (SSL) has proven to be successful in overcoming labeling difficulties by leveraging unlabeled data. Previous SSL algorithms typically assume a balanced class distribution. However, real-world datasets are usually class-imbalanced, causing the performance of existing SSL algorithms to be seriously decreased. One essential reason is that pseudo-labels for unlabeled data are selected based on a fixed confidence threshold, resulting in low performance on minority classes. In this paper, we develop a simple yet effective framework, which only involves adaptive thresholding for different classes in SSL algorithms, and achieves remarkable performance improvement on more than twenty imbalance ratios. Specifically, we explicitly optimize the number of pseudo-labels for each class in the SSL objective, so as to simultaneously obtain adaptive thresholds and minimize empirical risk. Moreover, the determination of the adaptive threshold can be efficiently obtained by a closed-form solution. Extensive experimental results demonstrate the effectiveness of our proposed algorithms.}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

Class-Imbalanced Semi-Supervised Learning with Adaptive Thresholding

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Adsh Optimization Objective for Class-Imbalanced Semi-Supervised Learning

Knowl 2 — Closed-Form Solution for Pseudo-Labels in Adsh

Knowl 3 — Class Prediction Invariance Under Adaptive Scaling

Knowl 4 — Class Bias Computation Under Known and Unknown Unlabeled Distributions

Knowl 5 — Adsh Training Algorithm

Knowl 6 — Classification Accuracy on Imbalanced CIFAR-10 with Matched Imbalance

Knowl 7 — Robustness of Adsh Under Mismatched Imbalance Ratios

Knowl 8 — Classification Accuracy on Imbalanced SVHN and STL-10

Knowl 9 — Combining Adsh with Classifier Re-Training (cRT)

Knowl 10 — Theoretical Convergence Limitation of Adsh

References

Citation

BibTeX

Source Code

Access the Paper