Conditional Adversarial Domain Adaptation

Mingsheng LongZhangjie CaoJianmin WangMichael I. Jordan

article2017NeurIPS2,935 citations

Proposes Conditional Domain Adversarial Networks to overcome multimodal alignment failures in deep domain adaptation by conditioning adversarial discriminators on classifier predictions and prediction uncertainty.

Listen

The paper introduces Conditional Domain Adversarial Networks (CDANs) to improve unsupervised domain adaptation for deep classification models. Existing adversarial methods often fail to align source and target distributions when those distributions contain the multimodal structure typical of multi-class problems, and they treat all examples equally even when classifier predictions carry high uncertainty.

The work sets out to condition the adversarial domain discriminator on information from the classifier predictions themselves, thereby capturing cross-covariance between features and labels while prioritizing examples that are easier to transfer. The authors derive two conditioning mechanisms—multilinear conditioning that explicitly models joint feature–prediction statistics and entropy conditioning that down-weights uncertain predictions—then embed them in a minimax objective with a theoretical bound on target risk derived from domain-adaptation theory.

The approach was evaluated on five standard visual benchmarks (Office-31, ImageCLEF-DA, Office-Home, digit datasets, and VisDA-2017) using both AlexNet and ResNet-50 backbones. Across six transfer tasks on Office-31 with ResNet, CDAN+E reached 87.7 percent average accuracy, exceeding the previous best result by roughly one point and widening the margin on the hardest shifts such as Amazon-to-Webcam. Similar consistent gains appeared on the more challenging Office-Home dataset (65.8 percent average) and on synthetic-to-real adaptation, where CDAN+E remained competitive with specialized generative pixel-level methods while using a simpler discriminative architecture.

These results indicate that conditioning the discriminator on classifier outputs allows the model to align multimodal distributions more reliably and to reduce measured domain discrepancy (A-distance) below that achieved by unconditional adversarial baselines. The entropy term further encourages confident predictions on the target domain, producing an auxiliary semi-supervised effect without extra labels.

The main limitations are the need for the discriminator family to be sufficiently expressive and the modest extra computation required by the multilinear map (mitigated by randomized approximations). The reported gains are stable across random seeds and datasets, yet the magnitude of improvement is largest when source and target domains differ substantially; on already-similar domains the advantage shrinks. Practitioners facing visual recognition tasks with unlabeled target data can therefore adopt CDAN or CDAN+E as a drop-in replacement for prior adversarial adapters, provided they verify that the chosen backbone and hyper-parameters (particularly the trade-off weight λ) are tuned on a held-out validation split.

Tsinghua University University of California Berkeley adversarial machine learning network architecture svm Conditional Adversarial Domain Adaptation Conditional Domain Adversarial Network Multilinear Conditioning

arXiv: 1705.10667 thuml/CDAN

Paper: Unsupervised Domain Adaptation by Backpropagation, Yaroslav Ganin et al. (2015). Reading Ganin and Lempitsky's introduction of domain-adversarial training via gradient reversal provides the essential adversarial foundation that the source paper conditions and extends.
Paper: Adversarial Discriminative Domain Adaptation, Eric Tzeng et al. (2017). Tzeng et al.'s Adversarial Discriminative Domain Adaptation establishes the foundational architectural split between source and target feature encoders that the source paper builds upon.
Paper: Conditional Generative Adversarial Nets, Mehdi Mirza et al. (2014). Mirza and Osindero's conditional generative adversarial networks provide the core conditioning concept that the source paper adapts for domain adaptation.
Paper: A theory of learning from different domains, Shai Ben-David et al. (2010). Ben-David et al.'s learning theory of domain adaptation provides the theoretical error bounds and divergence measures that motivate domain-invariant feature alignment.

Paper: CyCADA: Cycle-Consistent Adversarial Domain Adaptation, Judy Hoffman et al. (2018). CyCADA extends the source paper's adversarial feature alignment by integrating pixel-level translation and cycle-consistency constraints for cross-domain visual tasks.
Paper: StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation, Yunjey Choi et al. (2018). StarGAN builds on conditional adversarial principles to scale image-to-image translation across multiple domains using a single unified model.

Cover for Conditional Adversarial Domain Adaptation

Abstract

Adversarial learning has been embedded into deep networks to learn disentangled and transferable representations for domain adaptation. Existing adversarial domain adaptation methods may not effectively align different domains of multimodal distributions native in classification problems. In this paper, we present conditional adversarial domain adaptation, a principled framework that conditions the adversarial adaptation models on discriminative information conveyed in the classifier predictions. Conditional domain adversarial networks (CDANs) are designed with two novel conditioning strategies: multilinear conditioning that captures the cross-covariance between feature representations and classifier predictions to improve the discriminability, and entropy conditioning that controls the uncertainty of classifier predictions to guarantee the transferability. With theoretical guarantees and a few lines of codes, the approach has exceeded state-of-the-art results on five datasets.

Topics

adversarial machine learning network architecture svm transfer learning

Keywords

Conditional Adversarial Domain Adaptation Conditional Domain Adversarial Network Multilinear Conditioning Entropy Conditioning Domain Discriminator Cross‑Covariance Multimodal Distribution Unsupervised Domain Adaptation Transferability Theoretical Guarantee

1 Introduction
2 Related Work
3 Conditional Adversarial Domain Adaptation
3.1 Conditional Discriminator
3.2 Multilinear Conditioning
3.3 Conditional Domain Adversarial Network
3.4 Generalization Error Analysis
4 Experiments
4.1 Setup
4.2 Results
4.3 Analysis
5 Conclusion
References

Knowls

Knowl 1 — Conditional Domain Adversarial Network Minimax Formulation
model/method
In unsupervised domain adaptation from a labeled source domain $\mathcal{D}_s = \{(\mathbf{x}_i^s, y_i^s)\}_{i=1}^{n_s}$ drawn from joint distribution $P(\mathbf{x}^s, y^s)$ to an unlabeled target domain $\mathcal{D}_t = \{\mathbf{x}_j^t\}_{j=1}^{n_t}$ drawn from joint distribution $Q(\mathbf{x}^t, y^t)$ , Conditional Domain Adversarial Networks (CDAN) align multimodal data distributions across domains by conditioning the adversarial domain discriminator on classifier predictions.

Let $\mathbf{f} = F(\mathbf{x}) \in \mathbb{R}^{d_f}$ denote the feature representation extracted by a deep feature extractor $F$ , and $\mathbf{g} = G(\mathbf{x}) \in [0, 1]^C$ denote the softmax class probability prediction output by classifier $G$ across $C$ categories. Let $T(\mathbf{h})$ denote a conditioning map applied to the joint variable $\mathbf{h} = (\mathbf{f}, \mathbf{g})$ . A conditional domain discriminator $D$ takes $T(\mathbf{h})$ as input and outputs the probability that the joint representation originates from the source domain.

CDAN is formulated as a two-player minimax game between the classifier $G$ (including feature extractor $F$ ) and the conditional discriminator $D$ :

$\min_G \mathcal{E}(G) - \lambda \mathcal{E}(D, G)$ $\min_D \mathcal{E}(D, G)$

where $\lambda > 0$ is a trade-off hyperparameter, and the source classification loss $\mathcal{E}(G)$ and domain discriminator error $\mathcal{E}(D, G)$ are defined as:

$\mathcal{E}(G) = \mathbb{E}_{(\mathbf{x}_i^s, y_i^s) \sim \mathcal{D}_s} \mathcal{L}(G(\mathbf{x}_i^s), y_i^s)$

$\mathcal{E}(D, G) = -\mathbb{E}_{\mathbf{x}_i^s \sim \mathcal{D}_s} \log \left[ D(T(\mathbf{h}_i^s)) \right] - \mathbb{E}_{\mathbf{x}_j^t \sim \mathcal{D}_t} \log \left[ 1 - D(T(\mathbf{h}_j^t)) \right]$

where $\mathcal{L}(\cdot, \cdot)$ is the standard multi-class cross-entropy loss, $\mathbf{h}_i^s = (F(\mathbf{x}_i^s), G(\mathbf{x}_i^s))$ , and $\mathbf{h}_j^t = (F(\mathbf{x}_j^t), G(\mathbf{x}_j^t))$ . Minimizing $\mathcal{E}(D, G)$ over $D$ trains the discriminator to distinguish domains over the joint representation space, while maximizing $\mathcal{E}(D, G)$ over $F$ and $G$ encourages domain-invariant and discriminative representations.
Knowl 2 — Multilinear and Randomized Multilinear Conditioning Maps
model/method
To capture the cross-covariance dependency between feature representations $\mathbf{f} \in \mathbb{R}^{d_f}$ and classifier predictions $\mathbf{g} \in \mathbb{R}^{d_g}$ (where $d_g = C$ is the number of classes), Conditional Domain Adversarial Networks (CDAN) utilize multilinear conditioning rather than simple feature-prediction concatenation $\mathbf{f} \oplus \mathbf{g}$ .

The exact multilinear map $T_\otimes(\mathbf{f}, \mathbf{g})$ is defined as the tensor outer product:

$T_\otimes(\mathbf{f}, \mathbf{g}) = \mathbf{f} \otimes \mathbf{g}$

which produces a vector of dimension $d_f \times d_g$ . Under linear representations and one-hot labels, the expectation $\mathbb{E}_{\mathbf{x}, \mathbf{y}}[\mathbf{x} \otimes \mathbf{y}] = \mathbb{E}[\mathbf{x} \mid y = 1] \oplus \dots \oplus \mathbb{E}[\mathbf{x} \mid y = C]$ computes the mean of each class-conditional distribution $P(\mathbf{x} \mid y)$ , thereby capturing multimodal distribution structures.

To prevent dimension explosion when $d_f \times d_g$ is large, CDAN employs a randomized multilinear map $T_\odot(\mathbf{f}, \mathbf{g}) \in \mathbb{R}^d$ of dimension $d \ll d_f \times d_g$ :

$T_\odot(\mathbf{f}, \mathbf{g}) = \frac{1}{\sqrt{d}} (R_f \mathbf{f}) \odot (R_g \mathbf{g})$

where $\odot$ is the element-wise (Hadamard) product, and $R_f \in \mathbb{R}^{d \times d_f}$ and $R_g \in \mathbb{R}^{d \times d_g}$ are random matrices sampled once before training and kept fixed. Each entry $R_{ij}$ follows an independent symmetric distribution with mean zero and univariance ( $\mathbb{E}[R_{ij}] = 0$ , $\mathbb{E}[R_{ij}^2] = 1$ ), such as a standard Gaussian or Uniform distribution.

The conditioning strategy $T(\mathbf{h})$ for joint variable $\mathbf{h} = (\mathbf{f}, \mathbf{g})$ is selected via a dimension threshold:

$T(\mathbf{h}) = \begin{cases} T_\otimes(\mathbf{f}, \mathbf{g}) = \mathbf{f} \otimes \mathbf{g}, & \text{if } d_f \times d_g \le 4096 \\ T_\odot(\mathbf{f}, \mathbf{g}) = \frac{1}{\sqrt{d}} (R_f \mathbf{f}) \odot (R_g \mathbf{g}), & \text{otherwise} \end{cases}$

where 4096 is chosen to match the typical maximum hidden unit dimension of deep network layers.
Knowl 3 — Expectation and Variance of Randomized Multilinear Map
theoretical result
Let $\mathbf{f}, \mathbf{f}' \in \mathbb{R}^{d_f}$ and $\mathbf{g}, \mathbf{g}' \in \mathbb{R}^{d_g}$ be feature vectors and prediction vectors, respectively. Let $T_\otimes(\mathbf{f}, \mathbf{g}) = \mathbf{f} \otimes \mathbf{g}$ denote the exact multilinear outer product map, and let $T_\odot(\mathbf{f}, \mathbf{g}) = \frac{1}{\sqrt{d}} (R_f \mathbf{f}) \odot (R_g \mathbf{g})$ denote the explicit randomized multilinear map of dimension $d$ , where entries of $R_f \in \mathbb{R}^{d \times d_f}$ and $R_g \in \mathbb{R}^{d \times d_g}$ are independently drawn from a symmetric distribution with $\mathbb{E}[R_{ij}] = 0$ and $\mathbb{E}[R_{ij}^2] = 1$ .

The expectation and variance of the inner product $\langle T_\odot(\mathbf{f}, \mathbf{g}), T_\odot(\mathbf{f}', \mathbf{g}') \rangle$ satisfy:

$\mathbb{E} \left[ \langle T_\odot(\mathbf{f}, \mathbf{g}), T_\odot(\mathbf{f}', \mathbf{g}') \rangle \right] = \langle \mathbf{f}, \mathbf{f}' \rangle \langle \mathbf{g}, \mathbf{g}' \rangle = \langle T_\otimes(\mathbf{f}, \mathbf{g}), T_\otimes(\mathbf{f}', \mathbf{g}') \rangle$

$\text{var} \left[ \langle T_\odot(\mathbf{f}, \mathbf{g}), T_\odot(\mathbf{f}', \mathbf{g}') \rangle \right] = \sum_{i=1}^d \beta(R_f^i, \mathbf{f}) \beta(R_g^i, \mathbf{g}) + C$

where $R_f^i$ and $R_g^i$ are the $i$ -th rows of $R_f$ and $R_g$ , $\beta(R_f^i, \mathbf{f}) = \frac{1}{d} \sum_{j=1}^{d_f} [f_j^2 (f_j')^2 \mathbb{E}[(R_{ij}^f)^4] + C']$ , $\beta(R_g^i, \mathbf{g}) = \frac{1}{d} \sum_{j=1}^{d_g} [g_j^2 (g_j')^2 \mathbb{E}[(R_{ij}^g)^4] + C']$ , and $C, C'$ are constants.

This proves that $T_\odot(\mathbf{f}, \mathbf{g})$ is an unbiased estimator of the multilinear outer product inner product, with variance depending only on the fourth-order moments $\mathbb{E}[(R_{ij}^f)^4]$ and $\mathbb{E}[(R_{ij}^g)^4]$ , which are constant for symmetric distributions with univariance.
Knowl 4 — Entropy Conditioning for Uncertainty-Aware Domain Adversarial Adaptation
model/method
In conditional adversarial domain adaptation, training instances with uncertain predictions can mislead the domain discriminator. To prioritize discriminator training on easy-to-transfer samples with confident predictions, entropy conditioning (CDAN+E) weights each sample based on the Shannon entropy of its classifier prediction vector.

For a classifier probability output $\mathbf{g} = [g_1, \dots, g_C]^\top$ over $C$ classes, prediction uncertainty is measured by the entropy:

$H(\mathbf{g}) = -\sum_{c=1}^C g_c \log g_c$

Each source and target example is assigned an entropy-aware importance weight $w(H(\mathbf{g}))$ defined as:

$w(H(\mathbf{g})) = 1 + e^{-H(\mathbf{g})}$

When a prediction is certain ( $H(\mathbf{g}) \approx 0$ ), $w(H(\mathbf{g})) \approx 2$ ; when the prediction is highly uncertain ( $H(\mathbf{g}) \gg 0$ ), $w(H(\mathbf{g})) \to 1$ .

The minimax optimization problem of CDAN with entropy conditioning (CDAN+E) is:

$\min_G \mathbb{E}_{(\mathbf{x}_i^s, y_i^s) \sim \mathcal{D}_s} \mathcal{L}(G(\mathbf{x}_i^s), y_i^s) + \lambda \left( \mathbb{E}_{\mathbf{x}_i^s \sim \mathcal{D}_s} w(H(\mathbf{g}_i^s)) \log [D(T(\mathbf{h}_i^s))] + \mathbb{E}_{\mathbf{x}_j^t \sim \mathcal{D}_t} w(H(\mathbf{g}_j^t)) \log [1 - D(T(\mathbf{h}_j^t))] \right)$

$\max_D \mathbb{E}_{\mathbf{x}_i^s \sim \mathcal{D}_s} w(H(\mathbf{g}_i^s)) \log [D(T(\mathbf{h}_i^s))] + \mathbb{E}_{\mathbf{x}_j^t \sim \mathcal{D}_t} w(H(\mathbf{g}_j^t)) \log [1 - D(T(\mathbf{h}_j^t))]$

where $\mathcal{L}(\cdot, \cdot)$ is the cross-entropy loss, $\mathbf{h} = (\mathbf{f}, \mathbf{g})$ , and $T(\cdot)$ is the multilinear or randomized multilinear conditioning map. Entropy weighting reinforces the entropy minimization principle and facilitates semi-supervised transfer on unlabeled target data.
Knowl 5 — Generalization Error Bound and Delta-Distance in CDAN
theoretical result
Let $\epsilon_P(G) = \mathbb{E}_{(\mathbf{f}, y) \sim P}[G(\mathbf{f}) \ne y]$ and $\epsilon_Q(G) = \mathbb{E}_{(\mathbf{f}, y) \sim Q}[G(\mathbf{f}) \ne y]$ denote the source and target domain risks of a hypothesis $G \in \mathcal{H}$ over feature representation space $\mathbf{f} = F(\mathbf{x})$ . Let $G^* = \arg\min_{G \in \mathcal{H}} (\epsilon_P(G) + \epsilon_Q(G))$ be the ideal joint hypothesis. The target risk is bounded by:

$\epsilon_Q(G) \le \epsilon_P(G) + [\epsilon_P(G^*) + \epsilon_Q(G^*)] + |\epsilon_P(G, G^*) - \epsilon_Q(G, G^*)|$

where $\epsilon_P(G_1, G_2) = \mathbb{E}_{(\mathbf{f}, y) \sim P}[G_1(\mathbf{f}) \ne G_2(\mathbf{f})]$ .

Defining proxy joint distributions $P_G = (\mathbf{f}, G(\mathbf{f}))_{\mathbf{f} \sim P(\mathbf{f})}$ and $Q_G = (\mathbf{f}, G(\mathbf{f}))_{\mathbf{f} \sim Q(\mathbf{f})}$ over the joint variable $(\mathbf{f}, \mathbf{g})$ , and the difference hypothesis space $\Delta = \{\delta = |\mathbf{g} - G^*(\mathbf{f})| : G^* \in \mathcal{H}\}$ , the $\Delta$ -distance is defined as:

$d_\Delta(P_G, Q_G) = \sup_{\delta \in \Delta} \left| \mathbb{E}_{(\mathbf{f}, \mathbf{g}) \sim P_G}[\delta(\mathbf{f}, \mathbf{g}) \ne 0] - \mathbb{E}_{(\mathbf{f}, \mathbf{g}) \sim Q_G}[\delta(\mathbf{f}, \mathbf{g}) \ne 0] \right|$

This distance bounds the domain discrepancy:

$d_\Delta(P_G, Q_G) \ge |\epsilon_{P_G}(G^*) - \epsilon_{Q_G}(G^*)| = |\epsilon_P(G, G^*) - \epsilon_Q(G, G^*)|$

Assuming the domain discriminator hypothesis family $\mathcal{H}_D$ contains $\Delta$ ( $\Delta \subset \mathcal{H}_D$ ):

$d_\Delta(P_G, Q_G) \le \sup_{D \in \mathcal{H}_D} \left| \mathbb{E}_{(\mathbf{f}, \mathbf{g}) \sim P_G}[D(\mathbf{f}, \mathbf{g}) = 1] + \mathbb{E}_{(\mathbf{f}, \mathbf{g}) \sim Q_G}[D(\mathbf{f}, \mathbf{g}) = 0] \right|$

This supremum is achieved during the training of the optimal discriminator $D$ in CDAN, providing an empirical upper bound on $d_\Delta(P_G, Q_G)$ . Minimizing this bound with respect to representation $F$ bounds the target generalization risk $\epsilon_Q(G)$ .

Knowl 6 — Unsupervised Domain Adaptation Performance on Office-31

data/table

The Office-31 benchmark consists of 4,652 images across 31 object categories in three domains: Amazon (A), Webcam (W), and DSLR (D). Evaluation reports average classification accuracy (%) across three random trials on six transfer tasks using AlexNet and ResNet-50 backbones. Models are trained with SGD (momentum 0.9, initial learning rate $\eta_0 = 0.01$ , schedule $\eta_p = \eta_0(1 + 10p)^{-0.75}$ ), trade-off parameter $\lambda = 1$ , and progressive discriminator scheduling $\lambda_p = \frac{1 - \exp(-10p)}{1 + \exp(-10p)}$ .

Method	A $\rightarrow$ W	D $\rightarrow$ W	W $\rightarrow$ D	A $\rightarrow$ D	D $\rightarrow$ A	W $\rightarrow$ A	Avg
AlexNet	61.6 $\pm$ 0.5	95.4 $\pm$ 0.3	99.0 $\pm$ 0.2	63.8 $\pm$ 0.5	51.1 $\pm$ 0.6	49.8 $\pm$ 0.4	70.1
DAN	68.5 $\pm$ 0.5	96.0 $\pm$ 0.3	99.0 $\pm$ 0.3	67.0 $\pm$ 0.4	54.0 $\pm$ 0.5	53.1 $\pm$ 0.5	72.9
RTN	73.3 $\pm$ 0.3	96.8 $\pm$ 0.2	99.6 $\pm$ 0.1	71.0 $\pm$ 0.2	50.5 $\pm$ 0.3	51.0 $\pm$ 0.1	73.7
DANN	73.0 $\pm$ 0.5	96.4 $\pm$ 0.3	99.2 $\pm$ 0.3	72.3 $\pm$ 0.3	53.4 $\pm$ 0.4	51.2 $\pm$ 0.5	74.3
ADDA	73.5 $\pm$ 0.6	96.2 $\pm$ 0.4	98.8 $\pm$ 0.4	71.6 $\pm$ 0.4	54.6 $\pm$ 0.5	53.5 $\pm$ 0.6	74.7
JAN	74.9 $\pm$ 0.3	96.6 $\pm$ 0.2	99.5 $\pm$ 0.2	71.8 $\pm$ 0.2	58.3 $\pm$ 0.3	55.0 $\pm$ 0.4	76.0
CDAN	77.9 $\pm$ 0.3	96.9 $\pm$ 0.2	100.0 $\pm$ 0.0	75.1 $\pm$ 0.2	54.5 $\pm$ 0.3	57.5 $\pm$ 0.4	77.0
CDAN+E	78.3 $\pm$ 0.2	97.2 $\pm$ 0.1	100.0 $\pm$ 0.0	76.3 $\pm$ 0.1	57.3 $\pm$ 0.2	57.3 $\pm$ 0.3	77.7
ResNet-50	68.4 $\pm$ 0.2	96.7 $\pm$ 0.1	99.3 $\pm$ 0.1	68.9 $\pm$ 0.2	62.5 $\pm$ 0.3	60.7 $\pm$ 0.3	76.1
DAN	80.5 $\pm$ 0.4	97.1 $\pm$ 0.2	99.6 $\pm$ 0.1	78.6 $\pm$ 0.2	63.6 $\pm$ 0.3	62.8 $\pm$ 0.2	80.4
RTN	84.5 $\pm$ 0.2	96.8 $\pm$ 0.1	99.4 $\pm$ 0.1	77.5 $\pm$ 0.3	66.2 $\pm$ 0.2	64.8 $\pm$ 0.3	81.6
DANN	82.0 $\pm$ 0.4	96.9 $\pm$ 0.2	99.1 $\pm$ 0.1	79.7 $\pm$ 0.4	68.2 $\pm$ 0.4	67.4 $\pm$ 0.5	82.2
ADDA	86.2 $\pm$ 0.5	96.2 $\pm$ 0.3	98.4 $\pm$ 0.3	77.8 $\pm$ 0.3	69.5 $\pm$ 0.4	68.9 $\pm$ 0.5	82.9
JAN	85.4 $\pm$ 0.3	97.4 $\pm$ 0.2	99.8 $\pm$ 0.2	84.7 $\pm$ 0.3	68.6 $\pm$ 0.3	70.0 $\pm$ 0.4	84.3
GTA	89.5 $\pm$ 0.5	97.9 $\pm$ 0.3	99.8 $\pm$ 0.4	87.7 $\pm$ 0.5	72.8 $\pm$ 0.3	71.4 $\pm$ 0.4	86.5
CDAN	93.1 $\pm$ 0.2	98.2 $\pm$ 0.2	100.0 $\pm$ 0.0	89.8 $\pm$ 0.3	70.1 $\pm$ 0.4	68.0 $\pm$ 0.4	86.6
CDAN+E	94.1 $\pm$ 0.1	98.6 $\pm$ 0.1	100.0 $\pm$ 0.0	92.9 $\pm$ 0.2	71.0 $\pm$ 0.3	69.3 $\pm$ 0.3	87.7

CDAN+E achieves the highest average accuracy on both AlexNet (77.7%) and ResNet-50 (87.7%), outperforming unconditional adversarial methods (DANN, ADDA) and moment-matching methods (DAN, JAN). CDAN+E yields substantial improvements on difficult adaptation tasks with large domain discrepancy, including A $\rightarrow$ W (94.1% with ResNet-50 vs. 82.0% for DANN) and A $\rightarrow$ D (92.9% vs. 79.7% for DANN).

Knowl 7 — Unsupervised Domain Adaptation Performance on Office-Home

data/table

The Office-Home dataset contains 15,500 images across 65 categories from four visual domains: Artistic images (Ar), Clip Art (Cl), Product images (Pr), and Real-World images (Rw). Classification accuracies (%) across all 12 transfer tasks are reported using AlexNet and ResNet-50 backbones.

Method	Ar $\rightarrow$ Cl	Ar $\rightarrow$ Pr	Ar $\rightarrow$ Rw	Cl $\rightarrow$ Ar	Cl $\rightarrow$ Pr	Cl $\rightarrow$ Rw	Pr $\rightarrow$ Ar	Pr $\rightarrow$ Cl	Pr $\rightarrow$ Rw	Rw $\rightarrow$ Ar	Rw $\rightarrow$ Cl	Rw $\rightarrow$ Pr	Avg
AlexNet	26.4	32.6	41.3	22.1	41.7	42.1	20.5	20.3	51.1	31.0	27.9	54.9	34.3
DAN	31.7	43.2	55.1	33.8	48.6	50.8	30.1	35.1	57.7	44.6	39.3	63.7	44.5
DANN	36.4	45.2	54.7	35.2	51.8	55.1	31.6	39.7	59.3	45.7	46.4	65.9	47.3
JAN	35.5	46.1	57.7	36.4	53.3	54.5	33.4	40.3	60.1	45.9	47.4	67.9	48.2
CDAN	36.2	47.3	58.6	37.3	54.4	58.3	33.2	43.9	62.1	48.2	48.1	70.7	49.9
CDAN+E	38.1	50.3	60.3	39.7	56.4	57.8	35.5	43.1	63.2	48.4	48.5	71.1	51.0
ResNet-50	34.9	50.0	58.0	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
DAN	43.6	57.0	67.9	45.8	56.5	60.4	44.0	43.6	67.7	63.1	51.5	74.3	56.3
DANN	45.6	59.3	70.1	47.0	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
JAN	45.9	61.2	68.9	50.4	59.7	61.0	45.8	43.4	70.3	63.9	52.4	76.8	58.3
CDAN	49.0	69.3	74.5	54.4	66.0	68.4	55.6	48.3	75.9	68.4	55.4	80.5	63.8
CDAN+E	50.7	70.6	76.0	57.6	70.0	70.0	57.4	50.9	77.3	70.9	56.7	81.6	65.8

Office-Home embodies complex multimodal distributions due to its 65 categories and large inter-domain visual disparity. Category-agnostic methods struggle because features may align across domains without aligning class boundaries. CDAN and CDAN+E substantially outperform DANN (57.6% on ResNet-50) and JAN (58.3%), reaching 63.8% and 65.8% average accuracy respectively, demonstrating the effectiveness of conditioning on classifier predictions in complex multi-class domains.

Knowl 8 — Unsupervised Domain Adaptation Performance on ImageCLEF-DA

data/table

ImageCLEF-DA contains 12 object categories shared across three domains: Caltech-256 (C), ImageNet ILSVRC 2012 (I), and Pascal VOC 2012 (P). Classification accuracy (%) across all six cross-domain tasks is reported for AlexNet and ResNet-50 architectures.

Method	I $\rightarrow$ P	P $\rightarrow$ I	I $\rightarrow$ C	C $\rightarrow$ I	C $\rightarrow$ P	P $\rightarrow$ C	Avg
AlexNet	66.2 $\pm$ 0.2	70.0 $\pm$ 0.2	84.3 $\pm$ 0.2	71.3 $\pm$ 0.4	59.3 $\pm$ 0.5	84.5 $\pm$ 0.3	73.9
DAN	67.3 $\pm$ 0.2	80.5 $\pm$ 0.3	87.7 $\pm$ 0.3	76.0 $\pm$ 0.3	61.6 $\pm$ 0.3	88.4 $\pm$ 0.2	76.9
DANN	66.5 $\pm$ 0.6	81.8 $\pm$ 0.3	89.0 $\pm$ 0.4	79.8 $\pm$ 0.6	63.5 $\pm$ 0.5	88.7 $\pm$ 0.3	78.2
JAN	67.2 $\pm$ 0.5	82.8 $\pm$ 0.4	91.3 $\pm$ 0.5	80.0 $\pm$ 0.5	63.5 $\pm$ 0.4	91.0 $\pm$ 0.4	79.3
CDAN	67.7 $\pm$ 0.3	83.3 $\pm$ 0.1	91.8 $\pm$ 0.2	81.5 $\pm$ 0.2	63.0 $\pm$ 0.2	91.5 $\pm$ 0.3	79.8
CDAN+E	67.0 $\pm$ 0.4	84.8 $\pm$ 0.2	92.4 $\pm$ 0.3	81.3 $\pm$ 0.3	64.7 $\pm$ 0.3	91.6 $\pm$ 0.4	80.3
ResNet-50	74.8 $\pm$ 0.3	83.9 $\pm$ 0.1	91.5 $\pm$ 0.3	78.0 $\pm$ 0.2	65.5 $\pm$ 0.3	91.2 $\pm$ 0.3	80.7
DAN	74.5 $\pm$ 0.4	82.2 $\pm$ 0.2	92.8 $\pm$ 0.2	86.3 $\pm$ 0.4	69.2 $\pm$ 0.4	89.8 $\pm$ 0.4	82.5
DANN	75.0 $\pm$ 0.6	86.0 $\pm$ 0.3	96.2 $\pm$ 0.4	87.0 $\pm$ 0.5	74.3 $\pm$ 0.5	91.5 $\pm$ 0.6	85.0
JAN	76.8 $\pm$ 0.4	88.0 $\pm$ 0.2	94.7 $\pm$ 0.2	89.5 $\pm$ 0.3	74.2 $\pm$ 0.3	91.7 $\pm$ 0.3	85.8
CDAN	76.7 $\pm$ 0.3	90.6 $\pm$ 0.3	97.0 $\pm$ 0.4	90.5 $\pm$ 0.4	74.5 $\pm$ 0.3	93.5 $\pm$ 0.4	87.1
CDAN+E	77.7 $\pm$ 0.3	90.7 $\pm$ 0.2	97.7 $\pm$ 0.3	91.3 $\pm$ 0.3	74.2 $\pm$ 0.2	94.3 $\pm$ 0.3	87.7

CDAN+E achieves the top average performance across both backbones (80.3% on AlexNet, 87.7% on ResNet-50). Because the domains in ImageCLEF-DA are of equal size, class-balanced, and visually closer than Office-31, improvements over prior methods are consistent though smaller in magnitude.

Knowl 9 — Benchmark Results on Digits and VisDA-2017 Simulation-to-Real

data/table

Transfer performance is evaluated on three digit adaptation tasks trained from scratch (MNIST to USPS $\text{M}\rightarrow\text{U}$ , USPS to MNIST $\text{U}\rightarrow\text{M}$ , and SVHN to MNIST $\text{S}\rightarrow\text{M}$ ) and the large-scale VisDA-2017 Synthetic-to-Real benchmark (over 280,000 images across 12 classes) using ResNet-50.

Digits Benchmark (ResNet-50)					VisDA-2017 (ResNet-50)
Method	M $\rightarrow$ U	U $\rightarrow$ M	S $\rightarrow$ M	Avg	Method	Synthetic $\rightarrow$ Real
UNIT	96.0	93.6	90.5	93.4	JAN	61.6
CyCADA	95.6	96.5	90.4	94.2	GTA	69.5
CDAN	93.9	96.9	88.5	93.1	CDAN	66.8
CDAN+E	95.6	98.0	89.2	94.3	CDAN+E	70.0

CDAN+E matches or surpasses specialized generative pixel-level adaptation frameworks (UNIT, CyCADA, GTA), achieving 94.3% average accuracy on Digits and 70.0% on VisDA-2017 Synthetic $\rightarrow$ Real. Unlike pixel-level translation pipelines that require complex auxiliary generative architectures, CDAN+E achieves competitive transfer using a purely discriminative feature-level architecture.

Knowl 10 — Ablation of Random Matrix Sampling Strategies in CDAN

empirical result

In the randomized multilinear map $T_\odot(\mathbf{f}, \mathbf{g}) = \frac{1}{\sqrt{d}} (R_f \mathbf{f}) \odot (R_g \mathbf{g})$ , the sampling distribution of random projection matrices $R_f$ and $R_g$ affects transfer performance. Evaluated on the Office-31 benchmark using a ResNet-50 backbone, three variants of CDAN+E were compared:

CDAN+E (Gaussian sampling): Entries of $R_f$ and $R_g$ are drawn from a standard normal distribution $\mathcal{N}(0, 1)$ once and fixed.
CDAN+E (Uniform sampling): Entries of $R_f$ and $R_g$ are drawn from a zero-mean, unit-variance uniform distribution once and fixed.
CDAN+E (w/o random sampling): Directly uses the full outer product multilinear map $T_\otimes(\mathbf{f}, \mathbf{g}) = \mathbf{f} \otimes \mathbf{g}$ .

Method	A $\rightarrow$ W	D $\rightarrow$ W	W $\rightarrow$ D	A $\rightarrow$ D	D $\rightarrow$ A	W $\rightarrow$ A	Avg
CDAN+E (Gaussian sampling)	93.0 $\pm$ 0.2	98.4 $\pm$ 0.2	100.0 $\pm$ 0.0	89.2 $\pm$ 0.3	70.2 $\pm$ 0.4	67.4 $\pm$ 0.4	86.4
CDAN+E (Uniform sampling)	94.0 $\pm$ 0.2	98.4 $\pm$ 0.2	100.0 $\pm$ 0.0	89.8 $\pm$ 0.3	70.1 $\pm$ 0.4	69.4 $\pm$ 0.4	87.0
CDAN+E (w/o random sampling)	94.1 $\pm$ 0.1	98.6 $\pm$ 0.1	100.0 $\pm$ 0.0	92.9 $\pm$ 0.2	71.0 $\pm$ 0.3	69.3 $\pm$ 0.3	87.7

The exact multilinear map without random sampling achieves the highest average accuracy (87.7%). Among randomized approximations, uniform sampling (87.0%) outperforms Gaussian sampling (86.4%), and both randomized variants maintain high accuracy while reducing computational and parameter complexity.

Coverage note — None was omitted; all contributed methodology (CDAN objective, multilinear conditioning, randomized multilinear map, entropy conditioning), theoretical bounds (expectation, variance, generalization error), and empirical benchmarks on all five datasets (Office-31, Office-Home, ImageCLEF-DA, Digits, VisDA-2017) and ablations are included.

References

1.M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations (ICLR), 2017.
2.M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein gan. In International Conference on Machine Learning (ICML), 2017.
3.S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang. Generalization and equilibrium in generative adversarial nets (gans). In International Conference on Machine Learning (ICML), 2017.
4.S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. Machine Learning, 79(1-2):151–175, 2010.
5.S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems (NIPS), 2007.
6.Y. Bengio, A. Courville, and P. Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 35(8):1798–1828, 2013.
7.T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li. Mode regularized generative adversarial networks. International Conference on Learning Representations (ICLR), 2017.
8.Y. Chen, W. Chen, Y. Chen, B. Tsai, Y. F. Wang, and M. Sun. No more discrimination: Cross city adaptation of road scene segmenters. In The IEEE International Conference on Computer Vision (ICCV), pages 2011–2020, 2017.
9.R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research (JMLR), 12:2493–2537, 2011.
10.N. Courty, R. Flamary, A. Habrard, and A. Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. In Advances in Neural Information Processing Systems (NIPS), pages 3730–3739. 2017.
11.J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In International Conference on Machine Learning (ICML), 2014.
12.Y. Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning (ICML), 2015.
13.Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research (JMLR), 17(1):2096–2030, 2016.
14.X. Glorot, A. Bordes, and Y. Bengio. Domain adaptation for large-scale sentiment classification: A deep learning approach. In International Conference on Machine Learning (ICML), 2011.
15.B. Gong, K. Grauman, and F. Sha. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In International Conference on Machine Learning (ICML), 2013.
16.B. Gong, Y. Shi, F. Sha, and K. Grauman. Geodesic flow kernel for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
17.I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS), 2014.
18.R. Gopalan, R. Li, and R. Chellappa. Domain adaptation for object recognition: An unsupervised approach. In IEEE International Conference on Computer Vision (ICCV), 2011.
19.Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems (NIPS), 2005.
20.K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
21.J. Hoffman, S. Guadarrama, E. Tzeng, R. Hu, J. Donahue, R. Girshick, T. Darrell, and K. Saenko. LSDA: Large scale detection through adaptation. In Advances in Neural Information Processing Systems (NIPS), 2014.
22.J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell. CyCADA: Cycle-consistent adversarial domain adaptation. In Proceedings of the 35th International Conference on Machine Learning, pages 1989–1998, 2018.
23.J. Hoffman, D. Wang, F. Yu, and T. Darrell. Fcns in the wild: Pixel-level adversarial and constraint-based adaptation. CoRR, abs/1612.02649, 2016.
24.J. Huang, A. J. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf. Correcting sample selection bias by unlabeled data. In Advances in Neural Information Processing Systems (NIPS), 2006.
25.P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros. Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
26.P. Kar and H. Karnick. Random feature maps for dot product kernels. In International Conference on Artificial Intelligence and Statistics (AISTATS), volume 22, pages 583–591, 2012.
27.A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS), 2012.
28.M.-Y. Liu, T. Breuel, and J. Kautz. Unsupervised image-to-image translation networks. In Advances in Neural Information Processing Systems (NIPS), pages 700–708. 2017.
29.M. Long, Y. Cao, J. Wang, and M. I. Jordan. Learning transferable features with deep adaptation networks. In International Conference on Machine Learning (ICML), 2015.
30.M. Long, J. Wang, and M. I. Jordan. Deep transfer learning with joint adaptation networks. In International Conference on Machine Learning (ICML), 2017.
31.M. Long, H. Zhu, J. Wang, and M. I. Jordan. Unsupervised domain adaptation with residual transfer networks. In Advances in Neural Information Processing Systems (NIPS), pages 136–144, 2016.
32.L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research (JMLR), 9(Nov):2579–2605, 2008.
33.Y. Mansour, M. Mohri, and A. Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In Conference on Computational Learning Theory (COLT), 2009.
34.M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
35.A. Odena, C. Olah, and J. Shlens. Conditional image synthesis with auxiliary classifier gans. In International Conference on Machine Learning (ICML), 2017.
36.M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.
37.S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks (TNN), 22(2):199–210, 2011.
38.S. J. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering (TKDE), 22(10):1345–1359, 2010.
39.J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence. Dataset shift in machine learning. The MIT Press, 2009.
40.A. Rahimi and B. Recht. Random features for large-scale kernel machines. In Advances in neural information processing systems (NIPS), pages 1177–1184, 2008.
41.O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. 2014.
42.K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains. In European Conference on Computer Vision (ECCV), 2010.
43.S. Sankaranarayanan, Y. Balaji, C. D. Castillo, and R. Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
44.L. Song, B. Boots, S. M. Siddiqi, G. J. Gordon, and A. Smola. Hilbert space embeddings of hidden markov models. In International Conference on Machine Learning (ICML), 2010.
45.L. Song and B. Dai. Robust low rank kernel embeddings of multivariate distributions. In Advances in Neural Information Processing Systems (NIPS), pages 3228–3236, 2013.
46.L. Song, K. Fukumizu, and A. Gretton. Kernel embeddings of conditional distributions: A unified kernel framework for nonparametric inference in graphical models. IEEE Signal Processing Magazine, 30(4):98–111, 2013.
47.L. Song, J. Huang, A. Smola, and K. Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In International Conference on Machine Learning (ICML), 2009.
48.M. Sugiyama, M. Krauledat, and K.-R. Muller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research (JMLR), 8(May):985–1005, 2007.
49.M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in Neural Information Processing Systems (NIPS), 2008.
50.Y. Tsai, W. Hung, S. Schulter, K. Sohn, M. Yang, and M. Chandraker. Learning to adapt structured output space for semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
51.E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell. Adversarial discriminative domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
52.E. Tzeng, J. Hoffman, N. Zhang, K. Saenko, and T. Darrell. Simultaneous deep transfer across domains and tasks. In IEEE International Conference on Computer Vision (ICCV), 2015.
53.H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
54.J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems (NIPS), 2014.
55.K. Zhang, B. Schölkopf, K. Muandet, and Z. Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning (ICML), 2013.
56.J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.

Citation

MLA

Long, M., et al. “Conditional Adversarial Domain Adaptation”. arXiv, 2017, https://doi.org/10.48550/arxiv.1705.10667.

APA

Long, M., Cao, Z., Wang, J., & Jordan, M. I. (2017). Conditional Adversarial Domain Adaptation. arXiv. https://doi.org/10.48550/arxiv.1705.10667

Chicago

Long, M., Z. Cao, J. Wang, and M. I. Jordan. 2017. “Conditional Adversarial Domain Adaptation”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1705.10667.

Harvard

Long, M. et al. (2017) “Conditional Adversarial Domain Adaptation”. arXiv. Available at: https://doi.org/10.48550/arxiv.1705.10667.

Vancouver

1. Long M, Cao Z, Wang J, Jordan MI (2017) Conditional Adversarial Domain Adaptation. https://doi.org/10.48550/arxiv.1705.10667

BibTeX

@misc{https://doi.org/10.48550/arxiv.1705.10667,
  doi = {10.48550/ARXIV.1705.10667},
  url = {https://arxiv.org/abs/1705.10667},
  author = {Long, Mingsheng and Cao, Zhangjie and Wang, Jianmin and Jordan, Michael I.},
  keywords = {Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {Conditional Adversarial Domain Adaptation},
  publisher = {arXiv},
  year = {2017},
  copyright = {arXiv.org perpetual, non-exclusive license}
}

Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

Conditional Adversarial Domain Adaptation

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Conditional Domain Adversarial Network Minimax Formulation

Knowl 2 — Multilinear and Randomized Multilinear Conditioning Maps

Knowl 3 — Expectation and Variance of Randomized Multilinear Map

Knowl 4 — Entropy Conditioning for Uncertainty-Aware Domain Adversarial Adaptation

Knowl 5 — Generalization Error Bound and Delta-Distance in CDAN

Knowl 6 — Unsupervised Domain Adaptation Performance on Office-31

Knowl 7 — Unsupervised Domain Adaptation Performance on Office-Home

Knowl 8 — Unsupervised Domain Adaptation Performance on ImageCLEF-DA

Knowl 9 — Benchmark Results on Digits and VisDA-2017 Simulation-to-Real

Knowl 10 — Ablation of Random Matrix Sampling Strategies in CDAN

References

Citation

BibTeX

Source Code

Access the Paper