Back

Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation

Shiqi YangYaxing WangKai WangShangling JuiJoost van de Weijer

article2022NeurIPS182 citations

Proposes a simple source-free domain adaptation method that adapts models without source data by optimizing an upper bound on prediction consistency to attract neighboring features and disperse distant ones, setting a new state of the art on benchmarks like VisDA.

Centre de Visió per ComputadorHuaweiNankai Universitycontrastive learninghigh-dimensional feature spaceclustering methods
arXiv: 2205.04183Albert0147/AaD_SFDA
Cover for Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation

Abstract

We propose a simple but effective source-free domain adaptation (SFDA) method. Treating SFDA as an unsupervised clustering problem and following the intuition that local neighbors in feature space should have more similar predictions than other features, we propose to optimize an objective of prediction consistency. This objective encourages local neighborhood features in feature space to have similar predictions while features farther away in feature space have dissimilar predictions, leading to efficient feature clustering and cluster assignment simultaneously. For efficient training, we seek to optimize an upper-bound of the objective resulting in two simple terms. Furthermore, we relate popular existing methods in domain adaptation, source-free domain adaptation and contrastive learning via the perspective of discriminability and diversity. The experimental results prove the superiority of our method, and our method can be adopted as a simple but strong baseline for future research in SFDA. Our method can be also adapted to source-free open-set and partial-set DA which further shows the generalization ability of our method. Code is available in https://github.com/Albert0147/AaD_SFDA.

Topics

contrastive learninghigh-dimensional feature spaceclustering methodsfeature vectorsparameter adaptationtransfer learninglocal binary patternstraining sample

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Method
  • 3.1 Attracting and Dispersing for Source-free Domain Adaptation
  • 3.2 Relation to Existing Works
  • 4 Experiments
  • 4.1 Results and Analysis
  • 5 Conclusion
  • Acknowledgement
  • References
  • Checklist

Knowls

  1. Knowl 1 — Attracting and Dispersing Objective and Tractable Upper Bound for SFDA

    model/method

    In Source-Free Domain Adaptation (SFDA), a model consists of a feature extractor f:X→Rhf: \mathcal{X} \to \mathbb{R}^h and a classifier g:Rh→RCg: \mathbb{R}^h \to \mathbb{R}^C. For a target sample xi∈Dtx_i \in \mathcal{D}_t, its feature representation is zi=f(xi)∈Rhz_i = f(x_i) \in \mathbb{R}^h and its class prediction vector is pi=δ(g(zi))∈RCp_i = \delta(g(z_i)) \in \mathbb{R}^C, where δ\delta denotes the softmax function and CC is the number of classes.

    The conditional probability that feature ziz_i shares predictions with feature zjz_j across the dataset of size NtN_t is defined as:

    pij=exp⁡(piTpj)∑k=1Ntexp⁡(piTpk)p_{ij} = \frac{\exp(p_i^T p_j)}{\sum_{k=1}^{N_t} \exp(p_i^T p_k)}

    Let CiC_i denote the neighborhood set of KK-nearest neighbors of ziz_i in feature space (measured by cosine similarity), and BiB_i denote the background set of non-neighbor samples. The Attracting-and-Dispersing (AaD) likelihood ratio objective is formulated as minimizing the negative log-likelihood:

    L~i(Ci,Bi)=−log⁡P(Ci∣θ)P(Bi∣θ)=−log⁡∏j∈Cipij∏m∈Bipim\tilde{L}_i(C_i, B_i) = -\log \frac{P(C_i | \theta)}{P(B_i | \theta)} = -\log \frac{\prod_{j \in C_i} p_{ij}}{\prod_{m \in B_i} p_{im}}

    Applying Jensen's inequality under the condition ∣Ci∣<∣Bi∣|C_i| < |B_i| and approximating the background set BiB_i with all other samples in the mini-batch yields a tractable upper-bound surrogate loss for each sample ii:

    Li(Ci,Bi)=−∑j∈CipiTpj+λ∑m∈BipiTpmL_i(C_i, B_i) = -\sum_{j \in C_i} p_i^T p_j + \lambda \sum_{m \in B_i} p_i^T p_m

    where λ>0\lambda > 0 is a trade-off hyperparameter. The first term enforces prediction consistency (attracting) within the local neighborhood CiC_i, while the second term disperses predictions across dissimilar samples BiB_i in the mini-batch to promote cluster separation and prediction diversity.

  2. Knowl 2 — Attracting and Dispersing Training Algorithm

    algorithm

    The Attracting and Dispersing (AaD) framework adapts a source-pretrained model to an unlabeled target domain Dt\mathcal{D}_t without accessing source samples. Target features and predictions are cached in a dynamic memory bank that is updated asynchronously batch-by-batch.

    Input: Unlabeled target dataset Dt\mathcal{D}_t, source-pretrained feature extractor ff, classifier gg, neighbor count KK, batch size bsbs, decay schedule for λ\lambda
    Output: Adapted target model parameters θ={f,g}\theta = \{f, g\}
    Initialize memory bank Mz∈RNt×hM_z \in \mathbb{R}^{N_t \times h} with features zi=f(xi)z_i = f(x_i) for all xi∈Dtx_i \in \mathcal{D}_t
    Initialize memory bank Mp∈RNt×CM_p \in \mathbb{R}^{N_t \times C} with predictions pi=δ(g(zi))p_i = \delta(g(z_i)) for all xi∈Dtx_i \in \mathcal{D}_t
    while adaptation training not converged do
        Sample mini-batch T={xi}i=1bsT = \{x_i\}_{i=1}^{bs} from Dt\mathcal{D}_t
        Compute current features zi=f(xi)z_i = f(x_i) and predictions pi=δ(g(zi))p_i = \delta(g(z_i)) for each xi∈Tx_i \in T
        Update entries corresponding to TT in MzM_z and MpM_p
        
        for each sample i∈Ti \in T do
            Retrieve KK-nearest neighbors CiC_i of ziz_i from MzM_z using cosine similarity
            Retrieve historical predictions {pj}j∈Ci\{p_j\}_{j \in C_i} from MpM_p
            Define background set Bi=T∖{xi}B_i = T \setminus \{x_i\}
            Compute sample loss Li(Ci,Bi)=−∑j∈CipiTpj+λ∑m∈BipiTpmL_i(C_i, B_i) = -\sum_{j \in C_i} p_i^T p_j + \lambda \sum_{m \in B_i} p_i^T p_m
        end for
        
        Compute batch loss L=1bs∑i∈TLi(Ci,Bi)\mathcal{L} = \frac{1}{bs} \sum_{i \in T} L_i(C_i, B_i)
        Update model parameters θ\theta by gradient descent to minimize L\mathcal{L}
    end while
    return adapted parameters θ\theta

  3. Knowl 3 — Dispersing Loss Decay Schedule and Unsupervised Parameter Selection via SND

    model/method

    In the AaD loss objective Li(Ci,Bi)=−∑j∈CipiTpj+λ∑m∈BipiTpmL_i(C_i, B_i) = -\sum_{j \in C_i} p_i^T p_j + \lambda \sum_{m \in B_i} p_i^T p_m, the dispersing term prevents feature representation collapse early in training. As adaptation proceeds and semantic clusters form, samples belonging to the same class appear in the mini-batch background set BiB_i, making a strong dispersing penalty detrimental. To mitigate this class-collision effect, the weighting parameter λ\lambda is decayed over iterations:

    λ=(1+10⋅itermax_iter)−β\lambda = \left(1 + 10 \cdot \frac{\text{iter}}{\text{max\_iter}}\right)^{-\beta}

    where iter\text{iter} is the current iteration step, max_iter\text{max\_iter} is the maximum number of iterations, and β≥0\beta \ge 0 is the decay exponent.

    The decay parameter β\beta is chosen completely unsupervisedly using Soft Neighborhood Density (SND). Higher SND values correlate with superior target adaptation quality. Unsupervised grid search over β∈{0,0.25,0.5,1,2,3,4,5,7}\beta \in \{0, 0.25, 0.5, 1, 2, 3, 4, 5, 7\} via SND selects β=0\beta = 0 for Office-Home, β=2\beta = 2 for Office-31, and β=5\beta = 5 for VisDA-C.

  4. Knowl 4 — Taxonomy of Domain Adaptation and Contrastive Objectives via Discriminability and Diversity

    theoretical result

    Domain adaptation, source-free adaptation, and self-supervised contrastive learning objectives can be decomposed into two fundamental goals: maximizing discriminability (intra-cluster alignment or low-entropy predictions) and maximizing diversity (inter-cluster separation, uniform class distributions, or feature space uniformity):

    1. Mutual Information Maximization (SHOT-IM): LMI=H(Y∣X)−H(Y)\mathcal{L}_{MI} = H(Y|X) - H(Y). Conditional entropy H(Y∣X)H(Y|X) minimizes cluster assignment uncertainty (discriminability), while negative marginal entropy −H(Y)-H(Y) enforces uniform class allocation (diversity).
    2. Batch Nuclear-Norm Maximization (BNM): LBNM=−∥P∥F−rank(P)\mathcal{L}_{BNM} = -\|P\|_F - \text{rank}(P). Maximizing the Frobenius norm ∥P∥F\|P\|_F increases prediction certainty (discriminability), and maximizing rank(P)\text{rank}(P) prevents representation collapse across classes (diversity).
    3. Neighborhood Clustering (NC, G-SFDA, NRC): LNC=−∑j∈Cig(WijpiTpj)+∑c=1CKL(pˉc∥qc)\mathcal{L}_{NC} = -\sum_{j \in C_i} g(W_{ij} p_i^T p_j) + \sum_{c=1}^C \text{KL}(\bar{p}_c \parallel q_c). Local neighbor alignment maximizes prediction consistency, while KL divergence to a uniform prior qc=1/Cq_c = 1/C preserves class diversity.
    4. InfoNCE Contrastive Learning: LInfoNCE=E[−f(x)Tf(y)/τ]+E[log⁡(e1/τ+∑ief(xi−)Tf(x)/τ)]\mathcal{L}_{\text{InfoNCE}} = \mathbb{E}[-f(x)^T f(y)/\tau] + \mathbb{E}[\log(e^{1/\tau} + \sum_i e^{f(x_i^-)^T f(x)/\tau})]. The positive pair alignment term enforces feature clustering, while the negative pair denominator forces spherical uniformity.
    5. Attracting and Dispersing (AaD): LAaD=−∑j∈CipiTpj+λ∑m∈BipiTpm\mathcal{L}_{AaD} = -\sum_{j \in C_i} p_i^T p_j + \lambda \sum_{m \in B_i} p_i^T p_m. The attracting term aligns neighborhood predictions in output space, while the dispersing term repels mini-batch background predictions to maintain prediction diversity without assuming a uniform prior class distribution.

  5. Knowl 5 — Experimental Protocol and Implementation Details for SFDA

    experimental setup

    The experimental evaluation uses three standard benchmark datasets:

    • Office-31: 3 domains (Amazon, Webcam, DSLR), 31 categories, 4,652 images.
    • Office-Home: 4 domains (Art, Clipart, Product, Real-World), 65 categories, 15,500 images.
    • VisDA-C 2017: 12-class synthetic-to-real recognition benchmark with 152k synthetic source images and 55k real target images.

    Architecture: ResNet-50 backbone for Office-31 and Office-Home; ResNet-101 backbone for VisDA-C. The classifier head consists of: Fully Connected Layer →\to Batch Normalization →\to Fully Connected Layer with Weight Normalization.

    Optimization: SGD with momentum 0.9, batch size 64. Learning rates are 10−310^{-3} for the backbone and 10−210^{-2} for the classification head on Office-31 and Office-Home (scaled down by a factor of 10 for VisDA-C). Training durations are 40 epochs for Office-31 and Office-Home, and 15 epochs for VisDA-C.

    Hyperparameters: Number of nearest neighbors NCi=3N_{C_i} = 3 for Office-31 and Office-Home, NCi=5N_{C_i} = 5 for VisDA-C. Results are reported as the average over three independent random runs.

  6. Knowl 6 — Benchmark Accuracy on Closed-Set SFDA

    data/table

    AaD outperforms existing source-present and source-free domain adaptation methods across standard closed-set benchmarks.

    Method Source-Free Office-31 (Avg) Office-Home (Avg) VisDA-C (Per-class)
    ResNet (Source Only) - 76.1 46.1 52.4
    CDAN X 87.7 65.8 75.9
    SRDC X 90.8 71.3 -
    RWOT X - - 84.0
    3C-GAN ✓ 89.6 - 81.6
    SHOT ✓ 88.6 71.8 82.9
    A2^2Net ✓ - 72.8 84.3
    G-SFDA ✓ - 71.3 85.4
    NRC ✓ 89.4 72.2 85.9
    HCL ✓ 89.8 - 83.5
    AaD (Ours) ✓ 89.9 72.7 88.0

    On VisDA-C, AaD achieves 88.0% per-class average accuracy, outperforming the previous state-of-the-art SFDA method NRC (85.9%) by +2.1% and matching or exceeding source-present DA methods (such as RWOT at 84.0%). On Office-Home and Office-31, AaD reaches 72.7% and 89.9% average accuracy respectively, performing competitively against multi-classifier models like A2^2Net without requiring auxiliary classifiers or adversarial training.

  7. Knowl 7 — Source-Free Open-Set Domain Adaptation Performance on Office-Home

    data/table

    In Source-Free Open-Set Domain Adaptation (SF-ODA), the target domain contains unknown categories absent from the source domain. Evaluation is based on known class accuracy (OS∗OS^*), unknown category accuracy (UNKUNK), and the harmonic mean of known and unknown accuracies (HOS=2⋅OS∗⋅UNKOS∗+UNKHOS = \frac{2 \cdot OS^* \cdot UNK}{OS^* + UNK}).

    SHOT AaD (Ours)
    Task OS∗OS^* UNKUNK HOSHOS OS∗OS^* UNKUNK HOSHOS
    Ar →\to Cl 67.0 28.0 39.5 50.7 66.4 57.6
    Ar →\to Pr 81.8 26.3 39.8 64.6 69.4 66.9
    Ar →\to Rw 87.5 32.1 47.0 73.1 66.9 69.9
    Cl →\to Ar 66.8 46.2 54.6 48.2 81.1 60.5
    Cl →\to Pr 77.5 27.2 40.2 59.5 63.5 61.4
    Cl →\to Rw 80.0 25.9 39.1 67.4 68.3 67.8
    Pr →\to Ar 66.3 51.1 57.7 47.3 82.4 60.1
    Pr →\to Cl 59.3 31.0 40.8 45.4 72.8 55.9
    Pr →\to Rw 85.8 31.6 46.2 68.4 72.8 70.6
    Rw →\to Ar 73.5 50.6 59.9 54.5 79.0 64.6
    Rw →\to Cl 65.3 28.9 40.1 49.0 69.6 57.5
    Rw →\to Pr 84.4 28.2 42.3 69.7 70.6 70.1
    Average 74.6 33.9 45.6 58.2 71.9 63.6

    AaD achieves an average HOSHOS of 63.6% across 12 Office-Home tasks using a ResNet-50 backbone, substantially outperforming SHOT (45.6% average HOSHOS, an improvement of +18.0%). While SHOT exhibits a bias toward known classes at the expense of unknown category recognition (UNK=33.9%UNK = 33.9\%), AaD maintains a balanced trade-off (UNK=71.9%UNK = 71.9\%, OS∗=58.2%OS^* = 58.2\%).

  8. Knowl 8 — Source-Free Partial-Set Domain Adaptation Performance on Office-Home

    data/table

    In Source-Free Partial-Set Domain Adaptation (SF-PDA), target domain classes are a strict subset of source domain categories.

    Method Ar→\toCl Ar→\toPr Ar→\toRe Cl→\toAr Cl→\toPr Cl→\toRe Pr→\toAr Pr→\toCl Pr→\toRe Re→\toAr Re→\toCl Re→\toPr Avg
    SHOT-IM 57.9 83.6 88.8 72.4 74.0 79.0 76.1 60.6 90.1 81.9 68.3 88.5 76.8
    SHOT 64.8 85.2 92.7 76.3 77.6 88.8 79.7 64.3 89.5 80.6 66.4 85.8 79.3
    AaD (Ours) 67.0 83.5 93.1 80.5 76.0 87.6 78.1 65.6 90.2 83.5 64.3 87.3 79.7

    Using a ResNet-50 backbone on Office-Home across 12 adaptation transfer tasks, AaD achieves an overall average accuracy of 79.7%, exceeding SHOT-IM (76.8%) and full SHOT (79.3%).

  9. Knowl 9 — Memory Bank Subsampling and Computational Runtime Efficiency

    empirical result

    To reduce computational and memory overhead when training on large target datasets, the memory bank can be maintained as a sliding-window FIFO buffer of fixed capacity M<NtM < N_t. In this setting, bsbs new feature embeddings and predictions computed in the current mini-batch are appended to the buffer while the oldest bsbs entries are discarded.

    On the VisDA-C benchmark (55k target images, ResNet-101 backbone):

    • Full Memory Bank (100% target samples cached): Runtime of 520.13 s/epoch, achieving 88.0% per-class accuracy.
    • Subsampled Memory Bank (10% capacity): Runtime of 490.21 s/epoch, achieving 87.6% per-class accuracy.
    • Subsampled Memory Bank (5% capacity): Runtime of 482.77 s/epoch, achieving 87.5% per-class accuracy.
    • Baseline SHOT (epoch-level pseudo-labeling over all data): Runtime of 618.82 s/epoch, achieving 82.9% per-class accuracy.

    Subsampling the memory bank to 5% of dataset size maintains 99.4% of the full model's accuracy while reducing training runtime from 520.13 s/epoch to 482.77 s/epoch, demonstrating scalability to large datasets without full-dataset feature retention.

Coverage note — None was omitted; all key contributions including the AaD loss formulation, the training algorithm, diversity decay scheduling via SND, the taxonomy relating DA and contrastive learning, closed-set, open-set, partial-set benchmark results, and runtime memory bank analyses are fully represented.

References

  1. 1.Silvia Bucci, Mohammad Reza Loghmani, and Tatiana Tommasi. On the effectiveness of image rotation for open set domain adaptation. In European Conference on Computer Vision, pages 422–438. Springer, 2020.
  2. 2.Jianlong Chang, Lingfeng Wang, Gaofeng Meng, Shiming Xiang, and Chunhong Pan. Deep adaptive image clustering. In ICCV, pages 5879–5887, 2017.
  3. 3.Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In ICML, pages 1081–1090, 2019.
  4. 4.Safa Cicek and Stefano Soatto. Unsupervised domain adaptation via regularized conditional alignment. In ICCV, pages 1416–1425, 2019.
  5. 5.Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. CVPR, 2020.
  6. 6.Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Fast batch nuclear-norm maximization and minimization for robust domain adaptation. arXiv preprint arXiv:2107.06154, 2021.
  7. 7.Zhijie Deng, Yucen Luo, and Jun Zhu. Cluster alignment with a teacher for unsupervised domain adaptation. In ICCV, pages 9944–9953, 2019.
  8. 8.Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman. With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. ICCV, 2021.
  9. 9.Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17(1):2096–2030, 2016.
  10. 10.Jacob Goldberger, Geoffrey E Hinton, Sam Roweis, and Russ R Salakhutdinov. Neighbourhood components analysis. NIPS, 17, 2004.
  11. 11.Ryan Gomes, Andreas Krause, and Pietro Perona. Discriminative clustering by regularized information maximization. In NIPS, 2010.
  12. 12.Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, pages 2066–2073. IEEE, 2012.
  13. 13.Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016.
  14. 14.Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama. Learning discrete representations via information maximizing self-augmented training. In ICML, pages 1558–1567, 2017.
  15. 15.Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. NeurIPS, 34, 2021.
  16. 16.Zhizhong Huang, Jie Chen, Junping Zhang, and Hongming Shan. Exploring non-contrastive representation learning for deep clustering. arXiv preprint arXiv:2111.11821, 2021.
  17. 17.Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
  18. 18.Xu Ji, Joao F Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In ICCV, pages 9865–9874, 2019.
  19. 19.Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for versatile domain adaptation. ECCV, 2020.
  20. 20.Jogendra Nath Kundu, Naveen Venkat, and R Venkatesh Babu. Universal source-free domain adaptation. CVPR, 2020.
  21. 21.Jogendra Nath Kundu, Naveen Venkat, Ambareesh Revanur, R Venkatesh Babu, et al. Towards inheritable models for open-set domain adaptation. In CVPR, pages 12376–12385, 2020.
  22. 22.Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht. Sliced wasserstein discrepancy for unsupervised domain adaptation. In CVPR, pages 10285–10295, 2019.
  23. 23.Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. Prototypical contrastive learning of unsupervised representations. In ICLR, 2021.
  24. 24.Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In CVPR, pages 9641–9650, 2020.
  25. 25.Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. In AAAI, 2021.
  26. 26.Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. ICML, 2020.
  27. 27.Jian Liang, Dapeng Hu, and Jiashi Feng. Domain adaptation with auxiliary target domain-oriented classifier. In CVPR, pages 16632–16642, 2021.
  28. 28.Jian Liang, Dapeng Hu, Ran He, and Jiashi Feng. Distill and fine-tune: Effective adaptation from a black-box source model. CVPR, 2022.
  29. 29.Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. arXiv preprint arXiv:2012.07297, 2020.
  30. 30.Jian Liang, Dapeng Hu, Yunbo Wang, Ran He, and Jiashi Feng. Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  31. 31.Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self-training for domain adaptation. In NeurIPS, 2021.
  32. 32.Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Transferable representation learning with deep adaptation networks. TPAMI, 41(12):3071–3085, 2018.
  33. 33.Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. Learning transferable features with deep adaptation networks. ICML, 2015.
  34. 34.Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. In NIPS, pages 1647–1657, 2018.
  35. 35.Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Unsupervised domain adaptation with residual transfer networks. In NIPS, pages 136–144, 2016.
  36. 36.Zhihe Lu, Yongxin Yang, Xiatian Zhu, Cong Liu, Yi-Zhe Song, and Tao Xiang. Stochastic classifiers for unsupervised domain adaptation. In CVPR, pages 9111–9120, 2020.
  37. 37.Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  38. 38.Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. TKDE, 22(10):1345–1359, 2009.
  39. 39.Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924, 2017.
  40. 40.Simone Romano, James Bailey, Vinh Nguyen, and Karin Verspoor. Standardized mutual information for clustering comparisons: one step further in adjustment for chance. In ICML, pages 1143–1151, 2014.
  41. 41.Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV, pages 213–226. Springer, 2010.
  42. 42.Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. Universal domain adaptation through self supervision. NeurIPS, 33, 2020.
  43. 43.Kuniaki Saito, Donghyun Kim, Piotr Teterwak, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Tune it the right way: Unsupervised validation of domain adaptation via soft neighborhood density. In ICCV, pages 9184–9193, 2021.
  44. 44.Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR, pages 3723–3732, 2018.
  45. 45.Tim Salimans and Diederik P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. arXiv preprint arXiv:1602.07868, 2016.
  46. 46.Yuming Shen, Ziyi Shen, Menghan Wang, Jie Qin, Philip HS Torr, and Ling Shao. You never cluster alone. In NeurIPS, 2021.
  47. 47.Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. ICLR, 2018.
  48. 48.Jost Tobias Springenberg. Unsupervised and semi-supervised learning with categorical generative adversarial networks. In ICLR, 2015.
  49. 49.Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI, 2016.
  50. 50.Hui Tang, Ke Chen, and Kui Jia. Unsupervised domain adaptation via structurally regularized deep clustering. In CVPR, pages 8725–8735, 2020.
  51. 51.Tsung Wei Tsai, Chongxuan Li, and Jun Zhu. Mice: Mixture of contrastive experts for unsupervised image clustering. In ICLR, 2021.
  52. 52.Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, pages 7167–7176, 2017.
  53. 53.Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474, 2014.
  54. 54.Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In CVPR, pages 5018–5027, 2017.
  55. 55.Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. CVPR, 2022.
  56. 56.Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In ICML, pages 9929–9939. PMLR, 2020.
  57. 57.Ximei Wang, Liang Li, Weirui Ye, Mingsheng Long, and Jianmin Wang. Transferable attention for domain adaptation. In AAAI, volume 33, pages 5345–5352, 2019.
  58. 58.Jianlong Wu, Keyu Long, Fei Wang, Chen Qian, Cheng Li, Zhouchen Lin, and Hongbin Zha. Deep comprehensive correlation mining for image clustering. In ICCV, pages 8150–8159, 2019.
  59. 59.Yuan Wu, Diana Inkpen, and Ahmed El-Roby. Dual mixup regularized learning for adversarial domain adaptation. ECCV, 2020.
  60. 60.Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-parametric instance discrimination. In CVPR, pages 3733–3742, 2018.
  61. 61.Haifeng Xia, Handong Zhao, and Zhengming Ding. Adaptive adversarial network for source-free domain adaptation. In ICCV, pages 9010–9019, 2021.
  62. 62.Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang. Reliable weighted optimal transport for unsupervised domain adaptation. In CVPR, pages 4394–4403, 2020.
  63. 63.Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In ICCV, October 2019.
  64. 64.Shiqi Yang, Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neighborhood structure for source-free domain adaptation. NeurIPS, 34, 2021.
  65. 65.Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Unsupervised domain adaptation without source data by casting a bait. arXiv preprint arXiv:2010.12427, 2020.
  66. 66.Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In ICCV, pages 8978–8987, 2021.
  67. 67.Yabin Zhang, Bin Deng, Kui Jia, and Lei Zhang. Label propagation with augmented anchors: A simple semi-supervised learning baseline for unsupervised domain adaptation. In ECCV, pages 781–797, 2020.
  68. 68.Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan. Domain-symmetric networks for adversarial domain adaptation. In CVPR, pages 5031–5040, 2019.
  69. 69.Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In ICML, pages 7404–7413, 2019.
  70. 70.Chengxu Zhuang, Alex Lin Zhai, and Daniel Yamins. Local aggregation for unsupervised learning of visual embeddings. In ICCV, pages 6002–6012, 2019.

Citation

MLA
Yang, S., et al. “Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation”. arXiv, 2022, http://arxiv.org/abs/2205.04183v3.
APA
Yang, S., Wang, Y., Wang, K., Jui, S., & Weijer, J. van . de . (2022). Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation. arXiv. http://arxiv.org/abs/2205.04183v3
Chicago
Yang, S., Y. Wang, K. Wang, S. Jui, and J. van . de . Weijer. 2022. “Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation”. arXiv. http://arxiv.org/abs/2205.04183v3.
Harvard
Yang, S. et al. (2022) “Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2205.04183v3.
Vancouver
1. Yang S, Wang Y, Wang K, Jui S, Weijer J van de (2022) Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation. arXiv

BibTeX

@article{yang2022attracting,
  title = {Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation},
  author = {Yang, Shiqi and Wang, Yaxing and Wang, Kai and Jui, Shangling and Weijer, Joost van de},
  year = {2022},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2205.04183v3},
  eprint = {2205.04183}
}
Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: Authors