Back

Domain Adaptation for Time Series Under Feature and Label Shifts

Huan HeOwen QueenTeddy KokerConsuelo CuevasTheodoros TsiligkaridisMarinka Zitnik

article2023ICML147 citations

Presents RAINCOAT, an align-and-correct domain adaptation framework that jointly models time and frequency features to handle both distribution shifts and unseen target labels across diverse time series applications.

Listen

Deploying machine learning models on time series data across different real-world settings presents severe challenges due to domain shifts. Variations in sensor hardware, user behavior, and recording environments alter underlying data features across both the time and frequency domains. Furthermore, target deployment environments frequently contain category shifts—such as novel clinical diagnoses or unseen user activities—where target categories do not fully match the training data. Standard adaptation techniques fail in these scenarios because they ignore frequency-based shifts and cannot reliably detect novel, unobserved categories without labeled target data.

The article develops and evaluates RAINCOAT, an unsupervised domain adaptation framework tailored for complex time series that handles both feature shifts and label shifts. The primary objective is to demonstrate that aligning both temporal and frequency features while correcting for target-specific misalignments enables superior cross-domain generalization and robust identification of previously unseen categories across both closed-set and universal domain adaptation scenarios.

To accomplish this, the authors implemented an "align-then-correct" pipeline. The model first extracts joint temporal and frequency representations (using smoothing and Fourier transforms) and aligns them across domains using Sinkhorn divergence, an optimal transport metric well-suited for disjoint frequency supports. The model is then retrained on a target reconstruction task to adjust feature embeddings. If novel target categories exist, their embeddings drift significantly further from source reference points than common categories do. The framework was evaluated across five benchmark datasets encompassing human activity recognition, mechanical fault detection in industrial boilers, and sleep stage classification from brainwave recordings, comparing performance against 13 baseline domain adaptation methods using standardized convolutional backbones.

The experimental findings show that RAINCOAT consistently outperforms existing methods. In closed-set scenarios where classes match exactly, RAINCOAT achieved an average accuracy improvement of 6.77% and a macro-F1 improvement of 9.00% over the strongest baselines across all datasets, including absolute accuracy gains of up to 10.43% on industrial boiler fault detection and 8.65% on human activity recognition. In universal adaptation settings where target datasets contain novel categories, RAINCOAT improved harmonic mean accuracy by 16.33% over top baselines. Ablation analyses confirmed that adding the frequency encoder alone improved closed-set accuracy by 9.44%, while Sinkhorn alignment provided superior mathematical stability over traditional discrepancy measures without degrading performance when no label shifts were present.

These results demonstrate that explicitly modeling frequency features is vital for time series transfer learning, mitigating shortcut learning where models overfit to surface-level temporal patterns. For decision-makers and system architects, adopting this approach reduces operational risk, maintenance costs, and false predictions when deploying automated monitoring tools to new clinical sites or hardware configurations. Furthermore, the framework automatically identifies novel classes without manual threshold tuning, enabling reliable alerting when unexpected data patterns appear in production.

Organizations deploying automated time series monitoring should integrate dual-domain temporal and frequency alignment and adopt align-then-correct validation workflows to safely flag novel data. Future efforts should test this approach within source-free environments where source training data cannot be shared due to privacy regulations, as well as explore advanced transformer backbones and video adaptation.

While the findings demonstrate high reliability across diverse time series domains, stakeholders should note that frequency modeling offers the most substantial benefit on structured or periodic signals and may provide less leverage on highly irregular or non-stationary patterns. Overall confidence in the method's effectiveness is strongly supported by consistent empirical gains across multiple real-world benchmark tasks and baseline comparisons.

Harvard UniversityMassachusetts Institute of Technologytransfer learningtime series domain adaptationfeature shiftslabel shifts
arXiv: 2302.03133mims-harvard/Raincoat

No sufficiently relevant recommendations were found.

Cover for Domain Adaptation for Time Series Under Feature and Label Shifts

Abstract

Unsupervised domain adaptation (UDA) enables the transfer of models trained on source domains to unlabeled target domains. However, transferring complex time series models presents challenges due to the dynamic temporal structure variations across domains. This leads to feature shifts in the time and frequency representations. Additionally, the label distributions of tasks in the source and target domains can differ significantly, posing difficulties in addressing label shifts and recognizing labels unique to the target domain. Effectively transferring complex time series models remains a formidable problem. We present RAINCOAT, the first model for both closed-set and universal domain adaptation on complex time series. RAINCOAT addresses feature and label shifts by considering both temporal and frequency features, aligning them across domains, and correcting for misalignments to facilitate the detection of private labels. Additionally, RAINCOAT improves transferability by identifying label shifts in target domains. Our experiments with 5 datasets and 13 state-of-the-art UDA methods demonstrate that RAINCOAT can improve transfer learning performance by up to 16.33% and can handle both closed-set and universal domain adaptation.

Topics

transfer learning

Keywords

time series domain adaptationfeature shiftslabel shiftstime series classification

Table of Contents

  • 1. Introduction
  • 2. Related Work
  • 3. Problem Setup and Formulation
  • 4. Preliminaries
  • 5. RAINCOAT Approach
  • 5.1. Overview
  • 5.2. Time-Frequency Feature Encoder
  • 5.3. Domain Alignment of Time-Frequency Features
  • 5.4. Correction Step in RAINCOAT
  • 5.5. Inference: Detect Target Private Samples
  • 5.6. Overview of RAINCOAT Models
  • 6. Experiments
  • 6.1. Experimental Setup
  • 6.2. Results
  • 7. Conclusion
  • Acknowledgements
  • References
  • A. Further Information on Domain Alignment of Time-Frequency Feature
  • B. Details on Neural Networks RAINCOAT Algorithm
  • C. Additional Experimental Results
  • C.1. Dataset Details
  • C.2. Experimental Details
  • C.3. t-SNE Visualizations of Learned Representations for Closed-Set DA
  • C.4. Full Results of Closed-Set DA
  • C.5. Full Results of UniDA
  • C.6. Ablation Studies
  • D. Additional Discussion
  • D.1. The Use of Frequency Features
  • D.2. Extension to Video Domain Adaptation
  • D.3. Extension to Source-Free Domain Adaptation

Knowls

  1. Knowl 1 — RAINCOAT adapts time-series classifiers through alignment, correction, and drift-based rejection

    model/method

    RAINCOAT is an unsupervised domain-adaptation method for time-series classification with a labeled source domain and an unlabeled target domain. It uses a time-frequency encoder GTFG_{\mathrm{TF}}, a source-label classifier HH, and a reconstruction decoder UTFU_{\mathrm{TF}}. Its procedure has three stages: (1) train the encoder and classifier using source labels while aligning source and target representations and reconstructing source signals; (2) update the encoder and decoder using target-signal reconstruction, without source labels, to correct target representations; and (3) identify possible target-private classes from the movement of target representations during correction. In closed-set adaptation, target classes are assumed to be among the source classes; in universal adaptation, target-private classes can be rejected as unknown.

  2. Knowl 2 — The encoder combines smoothed Fourier features with learned temporal features

    model/method

    For each time series, RAINCOAT applies a cosine or Hann window before the discrete Fourier transform (DFT) to reduce frequency leakage caused by treating a finite segment as periodic. It applies a learned convolution to low-frequency Fourier modes and represents the resulting coefficients using their amplitude and phase. These frequency features are concatenated with features from a temporal encoder; the paper uses a CNN for the temporal branch. The joint representation is intended to expose domain-invariant frequency structure when temporal patterns shift, while retaining temporal information when frequency features are less informative. The paper notes that frequency features may be of limited value for non-periodic or non-stationary data, but argues that the joint representation can rely more on temporal features in such cases.

  3. Knowl 3 — RAINCOAT trains with source classification and reconstruction plus Sinkhorn domain alignment

    equation

    During alignment, RAINCOAT minimizes a sum of three losses on labeled source batches and unlabeled target batches:

    L=Lalign+Lrecon+Lcls.\mathcal{L}=\mathcal{L}_{\mathrm{align}}+\mathcal{L}_{\mathrm{recon}}+\mathcal{L}_{\mathrm{cls}}.

    Here, xsx^s is a source time-series batch with labels ysy^s, xtx^t is a target time-series batch, GTFG_{\mathrm{TF}} is the time-frequency encoder, UTFU_{\mathrm{TF}} is its decoder, and HH is the classifier. The alignment term is the Sinkhorn loss between GTF(xs)G_{\mathrm{TF}}(x^s) and GTF(xt)G_{\mathrm{TF}}(x^t); the reconstruction term measures the source-signal reconstruction error between xsx^s and UTF(GTF(xs))U_{\mathrm{TF}}(G_{\mathrm{TF}}(x^s)); and the classification term is cross-entropy between H(GTF(xs))H(G_{\mathrm{TF}}(x^s)) and ysy^s. Sinkhorn is an entropy-regularized optimal-transport measure that the authors select because it can compare distributions with disjoint supports. They argue that KL-based measures can become unstable under such shifts and that MMD can have weak or vanishing gradients when supports are far apart. The Sinkhorn regularization parameter is 10−310^{-3} in the reported experiments.

  4. Knowl 4 — Target-only reconstruction correction is intended to preserve common classes and separate unknown ones

    model/method

    After alignment, RAINCOAT records target representations zatz^t_{a} from the aligned encoder, then continues training the encoder and decoder on unlabeled target data using reconstruction loss alone. The resulting representations are denoted zctz^t_{c}. The motivation is the cluster assumption: samples in a cluster tend to share a label, so reconstructing target signals should preserve target-discriminative structure. The authors expect common target samples, which were aligned to source classes, to move less than target-private samples, which alignment may have placed near an incorrect source class. The classifier is not retrained in this correction stage; the representation change is later used to detect target-private samples.

  5. Knowl 5 — Target-private samples are detected from per-class representation drift

    algorithm

    For each target sample, RAINCOAT assigns a source class using the classifier and measures a cosine-similarity-based distance to that class's prototype before and after correction. Its drift score is the absolute difference between those two scores. It groups drift scores by predicted source class, applies a bimodality test to each group, and, when the test indicates two modes at p<0.05p<0.05, fits a two-means clustering to the scores. The centroid with the larger value is used as the rejection threshold for that class: samples in the higher-drift group are treated as target-private (unknown), while lower-drift samples are retained as common-class predictions. The procedure assumes that, when unknown target classes are present, the drift distribution can show two modes.

  6. Knowl 6 — The adaptation problem distinguishes feature shift from class-set and label-distribution shift

    definition

    Let ps(x,y)p_s(x,y) and pt(x,y)p_t(x,y) be the joint distributions of time-series inputs xx and labels yy in source and target domains. Feature shift is defined by ps(x)≠pt(x)p_s(x)\ne p_t(x); the paper describes the covariate-shift case as also having ps(y∣x)=pt(y∣x)p_s(y\mid x)=p_t(y\mid x). Label shift means ps(y)≠pt(y)p_s(y)\ne p_t(y) and can include changes in class proportions or differences in which classes occur. Write CsC_s and CtC_t for the source and target label sets. Closed-set adaptation assumes Cs=CtC_s=C_t and seeks accurate target predictions without access to target labels during training. Universal domain adaptation makes no such overlap assumption: common classes are Cs∩CtC_s\cap C_t, and target-private classes are Ct∖CsC_t\setminus C_s; it seeks predictions on common target classes while identifying target-private samples as unknown.

  7. Knowl 7 — Evaluation spans five time-series datasets and closed-set and universal settings

    experimental setup

    The evaluation covers human-activity datasets WISDM, HAR, and HHAR; the Boiler mechanical-fault dataset; and the Sleep-EDF EEG dataset. Participant, device, or other dataset-specific IDs define domains. Closed-set experiments use ten source-to-target domain pairs per dataset, except Boiler, for which all six possible directed configurations are evaluated. Universal-adaptation experiments include within-WISDM transfers and cross-dataset WISDM-to-HHAR and HHAR-to-WISDM transfers. Closed-set performance is measured with target-test accuracy and macro-F1. Universal performance is measured with H-score, the harmonic mean of accuracy on common classes and accuracy on target-private classes. The study compares against general and time-series closed-set adaptation methods and against UAN, DANCE, OVANet, and UniOT for universal adaptation.

  8. Knowl 8 — RAINCOAT improves closed-set results across all five evaluated datasets

    empirical result

    Across the five datasets, RAINCOAT has the highest reported average target accuracy. The values below compare RAINCOAT with the strongest baseline reported for each dataset:

    Could not parse LaTeX table

    The paper reports that RAINCOAT improves average accuracy by 6.77% and average macro-F1 by 9.00% over the strongest baseline across datasets. It also reports average gains over CLUDA of 8.23% in accuracy and 10.00% in macro-F1. These results are from target-domain evaluation; the accuracy values in the table are the reported dataset averages.

  9. Knowl 9 — RAINCOAT obtains higher H-scores in within- and cross-dataset universal adaptation

    empirical result

    Over five independent runs, RAINCOAT has the highest mean H-score in each reported universal-adaptation setting. H-scores are shown against the strongest competing method in each setting:

    Could not parse LaTeX table

    The paper reports a 16.33% average H-score improvement over the strongest baseline across these settings. H-score reflects the balance between recognizing common target classes and detecting target-private classes, rather than accuracy alone.

  10. Knowl 10 — Ablations associate frequency encoding, Sinkhorn alignment, and correction with complementary gains

    empirical result

    The WISDM ablation compares average accuracy across ten scenarios for closed-set adaptation and universal adaptation. The source-only model scores 65.78 and 40.84, respectively. Adding the frequency encoder (with MMD alignment when Sinkhorn is absent) gives 75.22 and 42.97. Using the frequency encoder with Sinkhorn gives 76.24 and 44.08; adding correction to the frequency-encoder configuration without Sinkhorn gives 76.04 and 46.52. The full configuration—frequency encoder, Sinkhorn alignment, and correction—scores 76.60 and 53.51. Thus, the full model has the best reported average in both settings; the correction component is particularly associated with improved universal-adaptation performance, while the paper reports no closed-set performance drop from including it. Comparing the frequency-encoder configurations with and without Sinkhorn, the paper reports gains of 1.02 percentage points for closed-set adaptation and 1.11 for universal adaptation.

Coverage note — The appendix's exploratory video-transfer experiment and proposed source-free extension are omitted because they are secondary to the evaluated time-series adaptation contribution; qualitative embedding visualizations are also omitted because they do not add a separate result beyond the reported metrics.

References

  1. 1.Suman Ravuri, Karel Lenc, Matthew Willson, Dmitry Kangin, Remi Lam, Piotr Mirowski, Megan Fitzsimons, Maria Athanassiadou, Sheleem Kashem, Sam Madge, et al. Skilful precipitation nowcasting using deep generative models of radar. Nature, 597(7878):672–677, 2021.
  2. 2.Scott M Lundberg, Bala Nair, Monica S Vavilala, Mayumi Horibe, Michael J Eisses, Trevor Adams, David E Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering, 2(10):749–760, 2018.
  3. 3.Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised contrastive pre-training for time series via time-frequency consistency. In Advances in Neural Information Processing Systems, 2022a.
  4. 4.Xiang Zhang, Marko Zeman, Theodoros Tsiligkaridis, and Marinka Zitnik. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Representations, ICLR, 2022b.
  5. 5.Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning (ICML), 2021.
  6. 6.Yawei Luo, Liang Zheng, Tao Guan, Junqing Yu, and Yi Yang. Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2502–2511, 2018.
  7. 7.Kun Zhang, Bernhard Scholkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International Conference on Machine Learning, 2013.
  8. 8.Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks, 2016.
  9. 9.Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning transferable features with deep adaptation networks. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15. JMLR.org, 2015.
  10. 10.Angela Zhang, Lei Xing, James Zou, and Joseph C Wu. Shifting machine learning for healthcare from development to deployment and from models to data. Nature Biomedical Engineering, pages 1–16, 2022c.
  11. 11.Emily Alsentzer, Michelle M Li, Shilpa N Kobren, Undiagnosed Diseases Network, Isaac S Kohane, and Marinka Zitnik. Deep learning for diagnosing patients with rare genetic diseases. medRxiv, pages 2022–12, 2022.
  12. 12.Sana Tonekaboni, Shalmali Joshi, Melissa D. McCradden, and Anna Goldenberg. What clinicians want: Contextualizing explainable machine learning for clinical end use. In Finale Doshi-Velez, Jim Fackler, Ken Jung, David Kale, Rajesh Ranganath, Byron Wallace, and Jenna Wiens, editors, Proceedings of the 4th Machine Learning for Healthcare Conference, volume 106 of Proceedings of Machine Learning Research, pages 359–380. PMLR, 09–10 Aug 2019. URL https://proceedings.mlr.press/v106/tonekaboni19a.html.
  13. 13.Robert Geirhos, Jorn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, Nov 2020. ISSN 2522-5839. doi:10.1038/s42256-020-00257-z.
  14. 14.Alex J. DeGrave, Joseph D. Janizek, and Su-In Lee. Ai for radiographic covid-19 detection selects shortcuts over signal. Nature Machine Intelligence, 3(77):610–619, Jul 2021. ISSN 2522-5839. doi:10.1038/s42256-021-00338-7.
  15. 15.Zachary Chase Lipton, Yu-Xiang Wang, and Alex Smola. Detecting and correcting for label shift with black box predictors. ArXiv, abs/1802.03916, 2018.
  16. 16.Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018a.
  17. 17.Guoliang Kang, Lu Jiang, Yi Yang, and Alexander Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4888–4897, 2019a.
  18. 18.Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Universal domain adaptation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2715–2724, 2019.
  19. 19.Bo Fu, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Learning to detect open classes for universal domain adaptation. In European Conference on Computer Vision, 2020.
  20. 20.Alexander Brown, Nenad Tomasev, Jan Freyberg, Yuan Liu, Alan Karthikesalingam, and Jessica Schrouff. Detecting and preventing shortcut learning for fair medical ai using shortcut testing (short), 2022.
  21. 21.Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015.
  22. 22.Ozan Sener, Hyun Oh Song, Ashutosh Saxena, and Silvio Savarese. Learning transferrable representations for unsupervised domain adaptation. Advances in neural information processing systems, 29, 2016.
  23. 23.Yue Zhang, Shun Miao, Tommaso Mansi, and Rui Liao. Task driven generative modeling for unsupervised domain adaptation: Application to x-ray image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 599–607. Springer, 2018.
  24. 24.Christian S Perone, Pedro Ballester, Rodrigo C Barros, and Julien Cohen-Adad. Unsupervised domain adaptation for medical imaging segmentation with self-ensembling. NeuroImage, 194:1–11, 2019.
  25. 25.Alan Ramponi and Barbara Plank. Neural unsupervised domain adaptation in nlp—a survey. arXiv preprint arXiv:2006.00632, 2020.
  26. 26.Judy Hoffman, Eric Tzeng, Trevor Darrell, and Kate Saenko. Simultaneous deep transfer across domains and tasks. 2015 IEEE International Conference on Computer Vision (ICCV), pages 4068–4076, 2015.
  27. 27.Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167–7176, 2017.
  28. 28.Saeid Motiian, Quinn Jones, Seyed Iranmanesh, and Gianfranco Doretto. Few-shot adversarial domain adaptation. Advances in neural information processing systems, 30, 2017.
  29. 29.Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning, pages 1989–1998. Pmlr, 2018.
  30. 30.Artem Rozantsev, Mathieu Salzmann, and Pascal V. Fua. Beyond sharing weights for deep domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:801–814, 2016.
  31. 31.Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV 2016 Workshops, 2016.
  32. 32.Nicolas Courty, Remi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. Advances in Neural Information Processing Systems, 30, 2017.
  33. 33.Ievgen Redko, Nicolas Courty, Remi Flamary, and Devis Tuia. Optimal transport for multi-source domain adaptation under target shift. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 849–858. PMLR, 2019.
  34. 34.Junchi Yan, Xu-Cheng Yin, Weiyao Lin, Cheng Deng, Hongyuan Zha, and Xiaokang Yang. A short survey of recent advances in graph matching. Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, 2016.
  35. 35.Debasmit Das and C. S. George Lee. Unsupervised domain adaptation using regularized hyper-graph matching. 2018 25th IEEE International Conference on Image Processing (ICIP), pages 3758–3762, 2018.
  36. 36.Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4893–4902, 2019b.
  37. 37.Ankit Singh. Clda: Contrastive learning for semi-supervised domain adaptation. Advances in Neural Information Processing Systems, 34:5089–5101, 2021.
  38. 38.Shixiang Tang, Peng Su, Dapeng Chen, and Wanli Ouyang. Gradient regularized contrastive learning for continual domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2665–2673, 2021.
  39. 39.Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, and Wen Li. Deep reconstruction-classification networks for unsupervised domain adaptation. In European conference on computer vision, pages 597–613. Springer, 2016.
  40. 40.I-Hong Jhuo, Dong Liu, DT Lee, and Shih-Fu Chang. Robust visual domain adaptation with low-rank reconstruction. In 2012 IEEE conference on computer vision and pattern recognition, pages 2168–2175. IEEE, 2012.
  41. 41.S. Purushotham, Wilka Carvalho, Tanachat Nilanon, and Yan Liu. Variational recurrent adversarial deep domain adaptation. In ICLR, 2017.
  42. 42.Garrett Wilson, Janardhan Rao Doppa, and Diane Joyce Cook. Multi-source deep domain adaptation with weak supervision for time-series sensor data. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
  43. 43.Ruichu Cai, Jiawei Chen, Zijian Li, Wei Chen, Keli Zhang, Junjian Ye, Zhuozhang Li, Xiaoyan Yang, and Zhenjie Zhang. Time series domain adaptation via sparse associative structure alignment. ArXiv, abs/2205.03554, 2021.
  44. 44.Qiao Liu and Hui Xue. Adversarial spectral kernel matching for unsupervised time series domain adaptation. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 2744–2750. International Joint Conferences on Artificial Intelligence Organization, 8 2021a.
  45. 45.Felix Ott, David Rugamer, Lucas Heublein, Bernd Bischl, and Christopher Mutschler. Domain adaptation for time-series classification to mitigate covariate shift. Proceedings of the 30th ACM International Conference on Multimedia, 2022.
  46. 46.Xiaoyong Jin, Youngsuk Park, Danielle Maddix, Hao Wang, and Yuyang Wang. Domain adaptation for time series forecasting via attention sharing. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 10280–10297. PMLR, 17–23 Jul 2022.
  47. 47.Yilmazcan Ozyurt, Stefan Feuerriegel, and Ce Zhang. Contrastive learning for unsupervised domain adaptation of time series. ArXiv, abs/2206.06243, 2022.
  48. 48.Garrett Wilson, Janardhan Rao Doppa, and Diane J. Cook. Calda: Improving multi-source time series domain adaptation with contrastive adversarial learning, 2021.
  49. 49.Liang Chen, Yihang Lou, Jianzhong He, Tao Bai, and Min Deng. Evidential neighborhood contrastive learning for universal domain adaptation. In AAAI Conference on Artificial Intelligence, 2022a.
  50. 50.Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. Universal domain adaptation through self supervision. ArXiv, abs/2002.07953, 2020.
  51. 51.Guangrui Li, Guoliang Kang, Yi Zhu, Yunchao Wei, and Yi Yang. Domain consensus clustering for universal domain adaptation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9752–9761, 2021a.
  52. 52.Liang Chen, Qianjin Du, Yihang Lou, Jianzhong He, Tao Bai, and Min Deng. Mutual nearest neighbor contrast and hybrid prototype self-training for universal domain adaptation. In AAAI, 2022b.
  53. 53.Wanxing Chang, Ye Shi, Hoang Duong Tuan, and Jingya Wang. Unified optimal transport framework for universal domain adaptation. ArXiv, abs/2210.17067, 2022.
  54. 54.Shai Ben-David, John Blitzer, Koby Crammer, and Fernando C Pereira. Analysis of representations for domain adaptation. In NIPS, 2006.
  55. 55.Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando C Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79:151–175, 2010.
  56. 56.Jennifer R. Kwapisz, Gary M. Weiss, and Samuel A. Moore. Activity recognition using cell phone accelerometers. SIGKDD Explor. Newsl., 12(2):74–82, mar 2011.
  57. 57.Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021b. URL https://openreview.net/forum?id=c8P9NQVtmnO.
  58. 58.F.J. Harris. On the use of windows for harmonic analysis with the discrete fourier transform. Proceedings of the IEEE, 66:51–83, 1978.
  59. 59.J. Feydy, T. Sejourne, F-X. Vialard, S-I. Amari, A. Trouve, and G. Peyre. Interpolating between optimal transport and mmd using sinkhorn divergences. In AISTATS, 2019.
  60. 60.Olivier Chapelle and Alexander Zien. Semi-supervised classification by low density separation. In International Conference on Artificial Intelligence and Statistics, 2005.
  61. 61.Mingsheng Long, ZHANGJIE CAO, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018b.
  62. 62.Rui Shu, Hung Hai Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. ArXiv, abs/1802.08735, 2018.
  63. 63.David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, and Alexey Kurakin. Adamatch: A unified approach to semi-supervised learning and domain adaptation. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=Q5uh1Nvv5dm.
  64. 64.Qiao Liu and Hui Xue. Adversarial spectral kernel matching for unsupervised time series domain adaptation. pages 2744–2750, 08 2021b. doi:10.24963/ijcai.2021/378.
  65. 65.Mohamed Ragab, Emadeldeen Eldele, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen, Min Wu, Chee Keong Kwoh, and Xiaoli Li. Adatime: A benchmarking suite for domain adaptation on time series data. arXiv preprint arXiv:2203.08321, 2022.
  66. 66.Kuniaki Saito and Kate Saenko. Ovanet: One-vs-all network for universal domain adaptation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8980–8989, 2021.
  67. 67.D. Anguita, Alessandro Ghio, L. Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. A public domain dataset for human activity recognition using smartphones. In The European Symposium on Artificial Neural Networks, 2013.
  68. 68.Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor S. Prentow, Mikkel Baun Kjærgaard, Anind K. Dey, Tobias Sonne, and Mads Møller Jensen. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, 2015.
  69. 69.R. Shohet, M. Kandil, and J.J. McArthur. Simulated boiler data for fault detection and classification, 2019. URL https://dx.doi.org/10.21227/awav-bn36.
  70. 70.Ary L. Goldberger, Luis A. Nunes Amaral, L Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and Harry Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. Circulation, 2000.
  71. 71.Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in Neural Information Processing Systems, 26, 2013.
  72. 72.Marco Cuturi and Gabriel Peyre. A smoothed dual approach for variational wasserstein problems. SIAM Journal on Imaging Sciences, 9(1):320–343, 2016.
  73. 73.Nicolas Fournier and Arnaud Guillin. On the rate of convergence in wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162:707–738, 2013.
  74. 74.Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Scholkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012. URL http://jmlr.org/papers/v13/gretton12a.html.
  75. 75.A. Genevay, G. Peyre, and M. Cuturi. Learning generative models with sinkhorn divergences. In AISTATS, 2018.
  76. 76.Soheil Kolouri, Phillip E. Pope, Charles E. Martin, and Gustavo K. Rohde. Sliced wasserstein auto-encoders. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=H1xaJn05FQ.
  77. 77.Richard Sinkhorn. A relationship between arbitrary positive matrices and doubly stochastic matrices. Annals of Mathematical Statistics, 35:876–879, 1964.
  78. 78.James W. Cooley and John W. Tukey. An algorithm for the machine calculation of complex fourier series. Mathematics of Computation, 19:297–301, 1965.
  79. 79.Kamisetty Ramamohan Rao and Pat Yip. The Transform and Data Compression Handbook. CRC Press, Inc., USA, 2000. ISBN 0849336929.
  80. 80.Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research. PMLR, 17–23 Jul 2022.
  81. 81.Junsik Kim, Tae-Hyun Oh, Seokju Lee, Fei Pan, and In-So Kweon. Variational prototyping-encoder: One-shot learning with prototypical images. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9454–9462, 2019.
  82. 82.Min-Hung Chen, Zsolt Kira, Ghassan Al-Regib, Jaekwon Yoo, Ruxin Chen, and Jian Zheng. Temporal attentive alignment for large-scale video domain adaptation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6320–6329, 2019a.
  83. 83.Donghyun Kim, Yi-Hsuan Tsai, Bingbing Zhuang, Xiang Yu, Stan Sclaroff, Kate Saenko, and Manmohan Chandraker. Learning cross-modal contrastive features for video domain adaptation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 13598–13607, 2021.
  84. 84.Jin Chen, Xinxiao Wu, Lixin Duan, and Shenghua Gao. Domain adversarial reinforcement learning for partial domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 33:539–553, 2019b.
  85. 85.Huan He, Yuanzhe Xi, and Joyce C Ho. Fast and accurate tensor decomposition without a high performance computing machine. In 2020 IEEE International Conference on Big Data (Big Data), pages 163–170. IEEE, 2020.
  86. 86.Huan He, Shifan Zhao, Yuanzhe Xi, and Joyce Ho. Gda-am: On the effectiveness of solving min-imax optimization via anderson mixing. In International Conference on Learning Representations, 2022.
  87. 87.Huan He, Shifan Zhao, Yuanzhe Xi, and Joyce C Ho. Med-diff: Generating electronic health records using accelerated denoising diffusion model, 2023.
  88. 88.Difeng Cai, Yuliang Ji, Huan He, Qiang Ye, and Yuanzhe Xi. Autm flow: atomic unrestricted time machine for monotonic normalizing flows. In James Cussens and Kun Zhang, editors, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, volume 180 of Proceedings of Machine Learning Research, pages 266–274. PMLR, 01–05 Aug 2022.
  89. 89.Yuang Liu, Wei Zhang, and Jun Wang. Source-free domain adaptation for semantic segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1215–1224, 2021.
  90. 90.Jogendra Nath Kundu, Naveen Venkat, V. RahulM., and R. Venkatesh Babu. Universal source-free domain adaptation. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4543–4552, 2020.
  91. 91.Shiqi Yang, Yaxing Wang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8958–8967, 2021.
  92. 92.Yuecong Xu, Jianfei Yang, Haozhi Cao, Keyu Wu, Min Wu, and Zhenghua Chen. Source-free video domain adaptation by learning temporal consistency for action recognition. In European Conference on Computer Vision, 2022.

Citation

MLA
He, H., et al. “Domain Adaptation for Time Series Under Feature and Label Shifts”. International Conference on Machine Learning, vol. 202, 2023, pp. 12746–74, https://proceedings.mlr.press/v202/he23b.html.
APA
He, H., Queen, O., Koker, T., Cuevas, C., Tsiligkaridis, T., & Zitnik, M. (2023). Domain Adaptation for Time Series Under Feature and Label Shifts. International Conference on Machine Learning, 202, 12746–12774. https://proceedings.mlr.press/v202/he23b.html
Chicago
He, H., O. Queen, T. Koker, C. Cuevas, T. Tsiligkaridis, and M. Zitnik. 2023. “Domain Adaptation for Time Series Under Feature and Label Shifts”. International Conference on Machine Learning 202: 12746–74. https://proceedings.mlr.press/v202/he23b.html.
Harvard
He, H. et al. (2023) “Domain Adaptation for Time Series Under Feature and Label Shifts”, International Conference on Machine Learning. PMLR, pp. 12746–12774. Available at: https://proceedings.mlr.press/v202/he23b.html.
Vancouver
1. He H, Queen O, Koker T, Cuevas C, Tsiligkaridis T, Zitnik M (2023) Domain Adaptation for Time Series Under Feature and Label Shifts. In: International Conference on Machine Learning. PMLR, pp 12746–12774

BibTeX

@InProceedings{pmlr-v202-he23b,
  title = 	 {Domain Adaptation for Time Series Under Feature and Label Shifts},
  author =       {He, Huan and Queen, Owen and Koker, Teddy and Cuevas, Consuelo and Tsiligkaridis, Theodoros and Zitnik, Marinka},
  booktitle = 	 {Proceedings of the 40th International Conference on Machine Learning},
  pages = 	 {12746--12774},
  year = 	 {2023},
  editor = 	 {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
  volume = 	 {202},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {23--29 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v202/he23b/he23b.pdf},
  url = 	 {https://proceedings.mlr.press/v202/he23b.html},
  abstract = 	 {Unsupervised domain adaptation (UDA) enables the transfer of models trained on source domains to unlabeled target domains. However, transferring complex time series models presents challenges due to the dynamic temporal structure variations across domains. This leads to feature shifts in the time and frequency representations. Additionally, the label distributions of tasks in the source and target domains can differ significantly, posing difficulties in addressing label shifts and recognizing labels unique to the target domain. Effectively transferring complex time series models remains a formidable problem. We present RAINCOAT, the first model for both closed-set and universal domain adaptation on complex time series. RAINCOAT addresses feature and label shifts by considering both temporal and frequency features, aligning them across domains, and correcting for misalignments to facilitate the detection of private labels. Additionally, RAINCOAT improves transferability by identifying label shifts in target domains. Our experiments with 5 datasets and 13 state-of-the-art UDA methods demonstrate that RAINCOAT can improve transfer learning performance by up to 16.33% and can handle both closed-set and universal domain adaptation.}
}
Metadata:DOI registry

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: https://creativecommons.org/licenses/by/4.0/