Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

John BlitzerMark DredzeFernando C Pereira

article2007ACL2,433 citations

Shows how structural correspondence learning transfers sentiment classifiers across product domains and provides a practical measure for choosing transferable source domains.

Listen

Organizations increasingly rely on automated sentiment analysis to monitor customer feedback and summarize opinions across diverse product categories. However, language varies considerably from one domain to another; for example, terms expressing satisfaction with electronics differ substantially from those used for books. Manually annotating large, labeled datasets for every separate domain is prohibitively expensive and time-consuming. When models trained on one domain are applied to another without modification, their accuracy degrades sharply due to differing vocabulary.

The article evaluates practical methods for cross-domain adaptation in sentiment classification, specifically testing how well classifiers trained on one product type can transfer to another without full-scale manual relabeling. It also explores whether domain similarity can be measured in advance using unlabeled data to guide data-collection strategies.

To address these questions, the authors extended Structural Correspondence Learning, an algorithm that maps features from different domains into a shared representation by identifying linking terms called pivot features. The study enhanced pivot selection by choosing words that frequently appear in both domains and carry high predictive value for sentiment using mutual information. The approach was evaluated on a balanced dataset of 8,000 labeled Amazon reviews and thousands of unlabeled reviews across four distinct categories: books, DVDs, electronics, and kitchen appliances. The analysis compared standard transfer baselines against adapted models and tested the effect of adding a small target sample to refine the alignments.

The key findings demonstrate significant performance gains and strategic insights into domain adaptation. First, selecting pivot features using mutual information reduced cross-domain transfer error by an average of 36% relative to a standard supervised baseline, outperforming basic Structural Correspondence Learning, which achieved a 21% reduction. Second, incorporating a minimal set of just 50 labeled examples from the target domain allowed the model to correct misaligned features, cutting the average transfer error by 46% compared to the baseline. Third, the analysis showed that domain relatedness heavily dictates transfer performance: reviews cluster naturally into related pairs (books with DVDs, and electronics with kitchen appliances), with transfers between related domains yielding significantly higher accuracy. Finally, the study established that a statistical proxy distance metric computed entirely on unlabeled data correlates closely with actual classification loss, reliably predicting which domains will transfer well.

These findings indicate that organizations do not need to build large, annotated datasets for every single product category. Instead, teams can achieve high classification accuracy by maintaining labeled data for a representative core set of domains and adapting them using abundant unlabeled text. Furthermore, spending minimal effort to label just 50 examples per target area provides a cost-effective way to fix alignment errors that occur when transitioning between very distinct subjects, such as kitchenware and literature.

For practical implementation, teams should use the unsupervised similarity metric to select an optimal, diverse portfolio of source domains to annotate—for instance, choosing one media category and one appliance category rather than redundant types. When deploying classifiers to new target areas, organizations should budget for a light labeling effort of roughly 50 target instances to adjust the low-dimensional projection parameters.

The study's primary limitation is its evaluation on a small group of four consumer review categories using binary (positive/negative) classification rather than multi-point rating scales or complex ranking. While confidence in the reported error reductions is high for text domains of similar scale, stakeholders should validate the approach on their specific data types—particularly when dealing with highly specialized language or complex ranking requirements—before large-scale production rollout.

University of Pennsylvania sentiment classification classification models svm Domain Adaptation Domains

Blitzer et al (2007).pdf

Paper: Analysis of Representations for Domain Adaptation, Shai Ben-David et al. (2006). This foundational paper establishes the generalization bounds and A-distance divergence measures that motivate structural correspondence learning for domain adaptation.

Paper: A theory of learning from different domains, Shai Ben-David et al. (2010). This paper directly extends domain adaptation theory by introducing HΔH-divergence bounds for combining source and target representations.
Paper: Learning Transferable Features with Deep Adaptation Networks, Mingsheng Long et al. (2015). This work builds upon sentiment and domain adaptation techniques by introducing multi-kernel maximum mean discrepancy to align deep neural network layers.

Cover for Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

Abstract

Automatic sentiment classification has been extensively studied and applied in recent years. However, sentiment is expressed differently in different domains, and annotating corpora for every possible domain of interest is impractical. We investigate domain adaptation for sentiment classifiers, focusing on online reviews for different types of products. First, we extend to sentiment classification the recently-proposed structural correspondence learning (SCL) algorithm, reducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline. Second, we identify a measure of domain similarity that correlates well with the potential for adaptation of a classifier from one domain to another. This measure could for instance be used to select a small set of domains to annotate whose trained classifiers would transfer well to many other domains.

Topics

sentiment classification classification models svm

Keywords

Domain Adaptation Domains Classi Sentiment Domain Adaptation Features Data Source SCL

1 Introduction
2 Structural Correspondence Learning
2.1 Algorithm Overview
2.2 Selecting Pivots with Mutual Information
3 Dataset and Baseline
4 Experiments with SCL and SCL-MI
5 Correcting Misalignments
6 Measuring Adaptability
6.1 The A-distance
6.2 Unlabeled Adaptability Measurements
7 Related Work
8 Conclusion
Acknowledgements
References

Knowls

Knowl 1 — Structural Correspondence Learning with Mutual Information for Sentiment Adaptation
model/method
Structural Correspondence Learning with Mutual Information (SCL-MI) extends Structural Correspondence Learning (SCL) to domain adaptation for sentiment classification by modifying how pivot features are selected.

Given unlabeled data from both source and target domains and labeled data from the source domain:
1. Pivot Feature Selection: In standard SCL, pivots are chosen purely by frequency across domains. In SCL-MI, candidates must occur in more than 5 documents in each domain, but are selected based on having the highest mutual information with the source domain class label. This ensures that pivots are both shared across domains and predictive of sentiment (e.g., selecting sentiment-laden words such as a must, a wonderful, loved it, weak, don't waste, and awful rather than purely functional/frequent words like book, one, very, they).
2. Auxiliary Linear Predictors: For each of the $m$ selected pivot features, an auxiliary linear classifier is trained on the combined unlabeled source and target data to predict the occurrence of that pivot given all other non-pivot features. The resulting weight vectors $w_\ell \in \mathbb{R}^d$ for $\ell \in \{1, \dots, m\}$ (where $d$ is the total number of features) form the matrix $W = [w_1, \dots, w_m] \in \mathbb{R}^{d \times m}$ .
3. Low-Dimensional Projection: Singular Value Decomposition (SVD) is performed on $W$ , and the top $k$ left singular vectors form a projection matrix $\theta \in \mathbb{R}^{k \times d}$ (using $k = 50$ ). These principal predictors map feature vectors into a shared cross-domain latent sentiment space.
4. Augmented Training: Every instance feature vector $x \in \mathbb{R}^d$ is mapped to an augmented representation $\langle x, \theta x \rangle \in \mathbb{R}^{d+k}$ . A linear classifier is then trained on the augmented source labeled instances to minimize Huber loss using stochastic gradient descent.
Knowl 2 — SCL Misalignment Correction via Low-Dimensional Regularization on Target Data
model/method
When adapting between diverse domains, SCL can introduce spurious alignments (misaligning topic-specific distinctions in broad domains with sentiment distinctions in narrow domains). To correct these misalignments without overfitting, SCL-MI utilizes a small set of labeled target domain instances ( $N = 50$ ) by regularizing only the low-dimensional projection parameters.

Let $w_s \in \mathbb{R}^d$ and $v_s \in \mathbb{R}^k$ be the weights learned on the source domain for the original features and the SCL projection features $\theta x$ , respectively:

$\min_{w, v} \sum_i L(w^\top x_i + v^\top \theta x_i, y_i) + \lambda \|w\|^2 + \mu \|v\|^2$

where $L$ is the loss function, $\lambda = 10^{-4}$ , and $\mu = 0$ .

Given $N = 50$ labeled target domain instances $(x_j, y_j)$ , each instance is augmented with the prediction score of the source classifier as a feature. The target model parameters are then learned by solving:

$\min_{w, v} \sum_j L(w^\top x_j + v^\top \theta x_j, y_j) + \lambda \|w\|^2 + \mu \|v - v_s\|^2$

with regularization hyperparameters $\lambda = \mu = 10^{-1}$ .

Because $N=50$ is insufficient to reliably re-estimate the $d \approx 200{,}000$ original feature weights in $w$ , penalizing deviation in $w$ provides no benefit. However, adapting only the $k = 50$ projection weights in $v$ allows the model to learn to downweight or ignore misaligned projections while preserving valid cross-domain correspondences.
Knowl 3 — Proxy A-Distance for Measuring Domain Adaptability
model/method
To evaluate the domain adaptability between a candidate source domain $\mathcal{D}$ and target domain $\mathcal{D}'$ using solely unlabeled data, the $\mathcal{A}$ -distance over linear classifiers is computed on the SCL projection space.

The theoretical $\mathcal{A}$ -distance is defined as:

$d_{\mathcal{A}}(\mathcal{D}, \mathcal{D}') = 2 \sup_{A \in \mathcal{A}} |\Pr_{\mathcal{D}}[A] - \Pr_{\mathcal{D}'}[A]|$

where $\mathcal{A}$ is the family of subsets of $\mathbb{R}^k$ corresponding to positive half-spaces of linear classifiers.

The proxy $\mathcal{A}$ -distance is computed empirically as follows:
1. Project unlabeled samples from both domains into the $k$ -dimensional SCL representation space $\theta x$ .
2. Assign domain-indicator binary labels to each projected instance (source domain vs. target domain).
3. Train a linear classifier using stochastic gradient descent to minimize Huber loss in discriminating between the two domains.
4. Compute the average empirical per-instance Huber loss $\bar{L}_{\text{Huber}}$ on this domain discrimination task.
5. Define the proxy $\mathcal{A}$ -distance as:
$\text{Proxy } d_{\mathcal{A}} = 100 \times (1 - \bar{L}_{\text{Huber}})$

A proxy distance of 0 indicates the two domains are indistinguishable under the linear classifier on $\theta x$ , while 100 indicates complete separability. This unsupervised measure correlates strongly with empirical transfer adaptation loss, enabling source domain selection prior to labeling.
Knowl 4 — Amazon Multi-Domain Sentiment Classification Benchmark
experimental setup
The benchmark dataset for domain adaptation in sentiment classification consists of Amazon product reviews across four product categories:
- Books (B)
- DVDs (D)
- Electronics (E)
- Kitchen appliances (K)
Dataset Specifications:
- Labeled Data: 2,000 reviews per domain, strictly balanced with 1,000 positive (rating $> 3$ stars) and 1,000 negative (rating $< 3$ stars); 3-star reviews are discarded as ambiguous. Each domain is split into 1,600 training and 400 test instances.
- Unlabeled Data: Balanced crawls ranging from 3,685 instances (DVDs) to 5,945 instances (Kitchen appliances).
- Feature Extraction & Training: Unigrams and bigrams. Linear classification models are trained to minimize Huber loss via stochastic gradient descent.
- SCL Hyperparameters: Number of singular vectors $k = 50$ ; pivot selection threshold requiring occurrence in $> 5$ documents in both domains.

Knowl 5 — Pairwise Domain Adaptation Classification Accuracies

empirical result

Evaluation of domain adaptation across all 12 directed source-to-target pairs among Books (B), DVDs (D), Electronics (E), and Kitchen (K) demonstrates consistent improvements of SCL-MI over both standard SCL and an unadapted supervised linear baseline.

Target	Source	Baseline (%)	SCL (%)	SCL-MI (%)	In-Domain Gold (%)
Books	D	72.8	76.8	79.7	80.4
Books	E	70.7	75.4	75.4	80.4
Books	K	70.9	66.1	68.6	80.4
DVD	B	77.2	74.0	75.8	82.4
DVD	E	70.6	74.3	76.2	82.4
DVD	K	72.7	75.4	76.9	82.4
Electronics	B	70.8	77.5	75.9	84.4
Electronics	D	73.0	74.1	74.1	84.4
Electronics	K	82.7	83.7	86.8	84.4
Kitchen	B	74.5	78.7	78.9	87.7
Kitchen	D	74.0	79.4	81.4	87.7
Kitchen	E	84.0	84.4	85.9	87.7

Notable behaviors:

Books and DVDs form a natural cluster with high mutual adaptability, as do Electronics and Kitchen.
In the Kitchen $\rightarrow$ Electronics transfer, SCL-MI achieves 86.8%, exceeding the in-domain gold standard accuracy of 84.4% due to the regularizing effect and additional semantic coverage provided by unlabeled data.

Knowl 6 — Cross-Domain Transfer Loss and Relative Error Reduction

data/table

The performance of adaptation methods is evaluated by measuring the transfer loss (in-domain gold standard accuracy minus transfer accuracy), averaged across source domains for each target domain.

Target Domain	Base	Base + 50 Targ	SCL	SCL-MI	SCL-MI + 50 Targ
Books	8.9%	9.0%	7.4%	5.8%	4.4%
DVD	8.9%	8.9%	7.8%	6.1%	5.3%
Electronics	8.3%	8.5%	6.0%	5.5%	4.8%
Kitchen	10.2%	9.9%	7.0%	5.6%	5.1%
Average	9.1%	9.1%	7.1%	5.8%	4.9%

Baseline linear classification suffers an average adaptation loss of 9.1% across all domains.
SCL reduces average adaptation loss to 7.1% (a 21% relative reduction in error due to transfer).
SCL-MI reduces average adaptation loss to 5.8% (a 36% relative reduction in error due to transfer).
Adding 50 labeled target domain instances to SCL-MI with projection-weight regularization reduces transfer loss to 4.9%, achieving an overall 46% relative error reduction over the baseline.
In contrast, adding 50 target instances to the baseline model without SCL features yields no improvement (9.1% loss) because 50 instances are insufficient to adjust weights across the full feature space.

Knowl 7 — Correlation Between Proxy A-Distance and Transfer Adaptation Loss
empirical result
The unsupervised proxy $\mathcal{A}$ -distance between domain pairs correlates strongly and linearly with the baseline adaptation loss between them:
- Electronics and Kitchen (EK): Lowest proxy $\mathcal{A}$ -distance ( $\,\approx 65$ ), lowest adaptation loss ( $\,\approx 2.5\%$ ).
- Books and DVDs (BD): Moderate proxy $\mathcal{A}$ -distance ( $\,\approx 81$ ), moderate adaptation loss ( $\,\approx 6.5\%$ ).
- DVDs and Electronics (DE): Higher proxy $\mathcal{A}$ -distance ( $\,\approx 87$ ), higher adaptation loss ( $\,\approx 9.0\%$ ).
- DVDs and Kitchen (DK): High proxy $\mathcal{A}$ -distance ( $\,\approx 89$ ), high adaptation loss ( $\,\approx 11.5\%$ ).
- Books and Electronics / Books and Kitchen (BE, BK): Highest proxy $\mathcal{A}$ -distance ( $\,\approx 95$ ), highest adaptation loss ( $\,\approx 11.5\%$ ).
This strong monotonic relationship allows practitioners to determine optimal source domains to annotate (e.g., choosing one text domain from Books/DVDs and one product domain from Electronics/Kitchen) using purely unlabeled data.
Knowl 8 — Asymmetric Semantic Breadth Causes Misalignment in Cross-Domain Projections
limitation
When transferring classifiers between domains with asymmetric semantic breadth (such as transferring from Kitchen appliances to Books), Structural Correspondence Learning can degrade performance compared to an unadapted baseline (e.g., K $\rightarrow$ B drops from 70.9% to 66.1% with SCL and 68.6% with SCL-MI).

This failure occurs because the broader source domain (Books) contains many latent projection dimensions that capture topical distinctions (such as religious vs. political texts) rather than sentiment polarity. When the narrower domain (Kitchen) is mapped into correspondence with these projections, uninformative topical dimensions in the target are assigned non-zero sentiment weights learned on the narrower source domain, thereby misaligning non-sentiment features and degrading target domain accuracy.

Coverage note — None was omitted. All principal contributions, methods (SCL-MI, target misalignment regularization, proxy A-distance), datasets, quantitative empirical tables/results, and analytical findings are covered.

References

1.Rie Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR, 6:1817–1853.
2.Anthony Aue and Michael Gamon. 2005. Customizing sentiment classifiers to new domains: a case study. http://research.microsoft.com/ anthaue/.
3.Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2006. Analysis of representations for domain adaptation. In Neural Information Processing Systems (NIPS).
4.John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In Empirical Methods in Natural Language Processing (EMNLP).
5.Ciprian Chelba and Alex Acero. 2004. Adaptation of maximum entropy capitalizer: Little data can help a lot. In EMNLP.
6.Sanjiv Das and Mike Chen. 2001. Yahoo! for amazon: Extracting market sentiment from stock message boards. In Proceedings of Athe Asia Pacific Finance Association Annual Conference.
7.R. Florian, H. Hassan, A.Ittycheriah, H. Jing, N. Kambhatla, X. Luo, N. Nicolov, and S. Roukos. 2004. A statistical model for multilingual entity detection and tracking. In of HLT-NAACL.
8.Andrew Goldberg and Xiaojin Zhu. 2004. Seeing stars when there aren’t many stars: Graph-based semisupervised learning for sentiment categorization. In HLT-NAACL 2006 Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing.
9.Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of Association for Computational Linguistics.
10.Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? sentiment classification using machine learning techniques. In Proceedings of Empirical Methods in Natural Language Processing.
11.Matt Thomas, Bo Pang, and Lillian Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. In Empirical Methods in Natural Language Processing (EMNLP).
12.Peter Turney. 2002. Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In Proceedings of Association for Computational Linguistics.
13.Tong Zhang. 2004. Solving large scale linear prediction problems using stochastic gradient descent algorithms. In International Conference on Machine Learning (ICML).

Citation

MLA

Blitzer, J., et al. “Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification”. Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 2007, pp. 440–47, https://aclanthology.org/P07-1056/.

APA

Blitzer, J., Dredze, M., & Pereira, F. (2007). Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 440–447. https://aclanthology.org/P07-1056/

Chicago

Blitzer, J., M. Dredze, and F. Pereira. 2007. “Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification”. Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 440–47. https://aclanthology.org/P07-1056/.

Harvard

Blitzer, J., Dredze, M. and Pereira, F. (2007) “Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification”, Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. Association for Computational Linguistics, pp. 440–447. Available at: https://aclanthology.org/P07-1056/.

Vancouver

1. Blitzer J, Dredze M, Pereira F (2007) Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. Association for Computational Linguistics, pp 440–447

BibTeX

@inproceedings{blitzer-etal-2007-biographies,
    title = "Biographies, {B}ollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification",
    author = "Blitzer, John  and
      Dredze, Mark  and
      Pereira, Fernando",
    editor = "Zaenen, Annie  and
      van den Bosch, Antal",
    booktitle = "Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics",
    month = jun,
    year = "2007",
    address = "Prague, Czech Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/P07-1056/",
    pages = "440--447"
}

Metadata:ACL Anthology

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by-nc-sa/4.0/

Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Structural Correspondence Learning with Mutual Information for Sentiment Adaptation

Knowl 2 — SCL Misalignment Correction via Low-Dimensional Regularization on Target Data

Knowl 3 — Proxy A-Distance for Measuring Domain Adaptability

Knowl 4 — Amazon Multi-Domain Sentiment Classification Benchmark