keyword
structural correspondence learning
Structural correspondence learning is a domain adaptation algorithm in machine learning that aligns feature spaces between a resource-rich source domain and a resource-scarce target domain to enable the effective transfer of predictive models. The method operates by selecting pivot features, which are common and behave consistently across both domains, and training auxiliary linear predictors to forecast the presence of these pivots using the remaining non-pivot features. By applying dimensionality reduction, such as singular value decomposition, to the learned weight vectors of these auxiliary predictors, the algorithm discovers a shared, low-dimensional representation that maps domain-specific features to common underlying concepts. This shared representation allows classifiers trained on labeled source domain data to generalize accurately to target domain data without requiring extensive target annotations.
4 items
Domain Adaptation with Structural Correspondence Learning
John Blitzer, Ryan T. McDonald, Fernando C Pereira
Why you should read this
Proposes Structural Correspondence Learning, an unsupervised domain adaptation framework that bridges vocabulary gaps across domains by aligning shared pivot features through auxiliary prediction tasks, significantly boosting transfer performance on part-of-speech tagging and parsing.
Discriminative learning methods are widely used in natural language processing. These methods work best when their training and test data are drawn from the same distribution. For many NLP tasks, however, we are confronted with new domains in which labeled data is scarce or non-existent. In such cases, we seek to adapt existing models from a resource-rich source domain to a resource-poor target domain. We introduce structural correspondence learning to automatically induce correspondences among features from different domains. We test our technique on part of speech tagging and show performance gains for varying amounts of source and target training data, as well as improvements in target domain parsing accuracy using our improved tagger.
Added
2026-09-24
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Xavier Glorot, Antoine Bordes, Yoshua Bengio
Why you should read this
Proposes an unsupervised deep learning framework based on stacked denoising auto-encoders that extracts domain-invariant representations to achieve state-of-the-art cross-domain sentiment classification on large-scale Amazon product reviews.
The exponential increase in the availability of online reviews and recommendations makes sentiment classification an interesting topic in academic and industrial research. Reviews can span so many different domains that it is difficult to gather annotated training data for all of them. Hence, this paper studies the problem of domain adaptation for sentiment classifiers, hereby a system is trained on labeled reviews from one source domain but is meant to be deployed on another. We propose a deep learning approach which learns to extract a meaningful representation for each review in an unsupervised fashion. Sentiment classifiers trained with this high-level feature representation clearly outperform state-of-the-art methods on a benchmark composed of reviews of 4 types of Amazon products. Furthermore, this method scales well and allowed us to successfully perform domain adaptation on a larger industrial-strength dataset of 22 domains.
Added
2026-09-18
Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification
John Blitzer, Mark Dredze, Fernando C Pereira
Why you should read this
Shows how structural correspondence learning transfers sentiment classifiers across product domains and provides a practical measure for choosing transferable source domains.
Automatic sentiment classification has been extensively studied and applied in recent years. However, sentiment is expressed differently in different domains, and annotating corpora for every possible domain of interest is impractical. We investigate domain adaptation for sentiment classifiers, focusing on online reviews for different types of products. First, we extend to sentiment classification the recently-proposed structural correspondence learning (SCL) algorithm, reducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline. Second, we identify a measure of domain similarity that correlates well with the potential for adaptation of a classifier from one domain to another. This measure could for instance be used to select a small set of domains to annotate whose trained classifiers would transfer well to many other domains.
Added
2026-09-14
Analysis of Representations for Domain Adaptation
Shai Ben-David, John Blitzer, K. Crammer, Fernando C Pereira
Why you should read this
Establishes a theoretical generalization bound for domain adaptation based on the A-distance between distributions, formalizing the fundamental trade-off between minimizing source classification error and learning aligned feature representations across domains.
Discriminative learning methods for classification perform well when training and test data are drawn from the same distribution. In many situations, though, we have labeled training data for a source domain, and we wish to learn a classifier which performs well on a target domain with a different distribution. Under what conditions can we adapt a classifier trained on the source domain for use in the target domain? Intuitively, a good feature representation is a crucial factor in the success of domain adaptation. We formalize this intuition theoretically with a generalization bound for domain adaption. Our theory illustrates the tradeoffs inherent in designing a representation for domain adaptation and gives a new justification for a recently proposed model. It also points toward a promising new model for domain adaptation: one which explicitly minimizes the difference between the source and target domains, while at the same time maximizing the margin of the training set.
Added
2026-09-14