Mining the peanut gallery: opinion extraction and semantic classification of product reviews

Kushal DaveSteve LawrenceDavid M. Pennock

article2003WWW2,483 citationsSeoul Test of Time Award

Proposes an opinion mining system that uses information retrieval scoring techniques and variable-length text patterns to classify review sentiment and synthesize unstructured web feedback into product attribute summaries.

Listen

The web hosts vast numbers of product reviews across dedicated sites, retailer platforms, and discussion forums, yet manually extracting an overall sense of sentiment on specific attributes remains time-consuming and impractical for users or companies. This challenge has grown with the explosion of online content and the rise of clipping services and review aggregators.

The article set out to build and evaluate automated methods for classifying reviews as positive or negative and then apply those methods to extract and summarize opinions from unstructured web search results.

The work trained classifiers on thousands of self-rated reviews from CNET and Amazon across electronics categories, testing performance with cross-category and balanced within-category splits. It compared information-retrieval techniques such as n-gram features, term-frequency scoring, and simple substitutions against standard machine-learning approaches including SVMs and Naive Bayes, then extended the best classifiers to sentences gathered from web searches.

N-gram methods with a straightforward bias-based score reached 85–88 percent accuracy on structured reviews, matching or exceeding machine-learning baselines; variable-length substring features and limited metadata substitutions provided modest further gains. Web-sentence classification proved far noisier, with many ambiguous or off-topic fragments, yet grouping sentences under simple attribute bigrams produced subjectively coherent summaries. Negative reviews were harder to recall because of greater language variety and data skew toward positives.

These results indicate that lightweight, domain-adaptable classifiers can automate much of the work now done by review hubs and clipping services, lowering the cost and time required for competitive intelligence and consumer research while highlighting persistent difficulties with ambivalence, short texts, and mixed signals.

Further progress requires finer-grained annotated corpora for sentence-level evaluation, larger numbers of test folds to reduce variance, efficiency improvements for substring algorithms, and an upstream genre filter to separate true review fragments from other subjective text.

Performance varied noticeably across test conditions, web-mining accuracy was assessed subjectively on a modest sample, and many refinements failed to generalize; readers should therefore treat the reported accuracy figures as indicative rather than definitive.

NEC Laboratories America, Inc.Overture Services, Inc.automation blogs competitive intelligence opinion extraction product reviews semantic classification

Dave et al (2003).pdf

Paper: Thumbs up? Sentiment Classification using Machine Learning Techniques, Bo Pang et al. (2002). This pioneering paper established machine-learning baselines such as SVMs and Naive Bayes for document-level sentiment classification on reviews, providing the direct foundation and comparative benchmarks for the source.
Paper: Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, Peter D. Turney (2002). This work introduced unsupervised semantic orientation extraction from web review data, establishing foundational principles for classifying product sentiment without full manual supervision.
Paper: Predicting the Semantic Orientation of Adjectives, V. Hatzivassiloglou et al. (1997). This seminal study introduced methods for predicting the semantic orientation of adjectives from text corpora, establishing core lexical polarity foundations used in opinion extraction.
Paper: Machine learning in automated text categorization, Fabrizio Sebastiani (2001). This comprehensive survey outlines standard text categorization techniques and document representations that form the technical basis of the classifiers evaluated in the source.

Paper: Mining and summarizing customer reviews, Minqing Hu et al. (2004). This work directly advances product review mining by establishing fine-grained, feature-based summarization to extract specific product attributes and their associated opinion polarities.
Paper: A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts, Bo Pang et al. (2004). This paper addresses the challenge of separating subjective opinions from objective text highlighted in the source by applying minimum-cut graph formulations for sentence-level subjectivity extraction.
Paper: Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification, John Blitzer et al. (2007). This study tackles the cross-category performance degradation identified in the source by developing structural correspondence learning for domain adaptation across product review types.
Paper: Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales, Bo Pang et al. (2005). This work extends binary positive-versus-negative review classification to multi-point rating scales using metric labeling to capture nuanced customer evaluations.
Paper: Attention-based LSTM for Aspect-level Sentiment Classification, Yequan Wang et al. (2016). This paper modernizes aspect-level opinion mining on review sentences by applying attention-based neural networks to associate sentiments directly with target product attributes.

Cover for Mining the peanut gallery: opinion extraction and semantic classification of product reviews

Abstract

The web contains a wealth of product reviews, but sifting through them is a daunting task. Ideally, an opinion mining tool would process a set of search results for a given item, generating a list of product attributes (quality, features, etc.) and aggregating opinions about each of them (poor, mixed, good). We begin by identifying the unique properties of this problem and develop a method for automatically distinguishing between positive and negative reviews. Our classifier draws on information retrieval techniques for feature extraction and scoring, and the results for various metrics and heuristics vary depending on the testing situation. The best methods work as well as or better than traditional machine learning. When operating on individual sentences collected from web searches, performance is limited due to noise and ambiguity. But in the context of a complete web-based tool and aided by a simple method for grouping sentences into attributes, the results are qualitatively quite useful.

discussion boards and mailing list archives, as well as in Usenet via Google Groups. Users also comment on products in their personal web sites and blogs, which are then aggregated by sites such as Blogstreet.com, AllConsuming.net, and onfocus.com. When trying to locate information on a product, a general web search turns up several useful sites, but getting an overall sense of these reviews can be daunting or time-consuming.

In the movie review domain, sites like Rottentomatoes.com have sprung up to try to impose some order on the void, providing ratings and brief quotes from numerous reviews and generating an aggregate opinion. Such sites even have their own category—“Review Hubs”—on Yahoo!

On the commercial side, Internet clipping services like Webclipping.com, eWatch.com, and TracerLock.com watch news sites and discussion areas for mentions of a given company or product, trying to track “buzz.” Print clipping services have been providing competitive intelligence for some time. The ease of publishing on the web led to an explosion in content to be surveyed, but the same technology makes automation much more feasible.

This paper describes a tool for sifting through and synthesizing product reviews, automating the sort of work done by aggregation sites or clipping services. We begin by using structured reviews for testing and training, identifying appropriate features and scoring methods from information retrieval for determining whether reviews are positive or negative. These results perform as well as traditional machine learning methods. We then use the classifier to identify and classify review sentences from the web, where classification is more difficult. However, a simple technique for identifying the relevant attributes of a product produces a subjectively useful summary.

Topics

automation blogs competitive intelligence feature extraction web searches classification models customer review web content support vector machine

Keywords

opinion extraction product reviews semantic classification opinion mining semantic orientation linguistic features Naive Bayes binary classification

1. INTRODUCTION
2. BACKGROUND
2.1 Objectivity classification
2.2 Word classification
2.3 Sentiment classification
2.3.1 Affect and direction
2.3.2 Recommendations
2.3.3 Commercial projects
3. APPROACH
3.1 Corpus
3.2 Evaluation
3.3 Feature selection
3.3.1 Metadata and statistical substitutions
3.3.2 Linguistic substitutions
3.3.3 Language-based modifications
3.3.4 N-grams and proximity
3.3.5 Substrings
3.4 Thresholding
3.5 Smoothing
3.6 Scoring
3.7 Reweighting
3.8 Classifying
3.9 Scalar ratings
3.10 Mining
3.10.1 Evaluation
3.10.2 Presentation
4. SUMMARY AND CONCLUSIONS
5. REFERENCES

Knowls

Knowl 1 — Normalized Frequency Bias Scoring and Classification Rule
model/method
The classifier assigns a polarity score to each term or n-gram feature $f_i$ based on the difference between its class-conditional probabilities across positive reviews $C$ and negative reviews $C'$ :

$score(f_i) = \frac{p(f_i \mid C) - p(f_i \mid C')}{p(f_i \mid C) + p(f_i \mid C')}$

where the class-conditional probability $p(f_i \mid C)$ is the normalized term frequency:

$p(f_i \mid C) = \frac{\text{count}(f_i, C)}{\sum_{f_k} \text{count}(f_k, C)}$

Here, $\text{count}(f_i, C)$ is the number of occurrences of feature $f_i$ in training documents belonging to class $C$ , and the denominator sums occurrences over all features in $C$ . The resulting $score(f_i) \in [-1, 1]$ represents the feature's bias toward positive ( $+1$ ) or negative ( $-1$ ) sentiment.

For a document $d = (f_1, f_2, \dots, f_n)$ consisting of $n$ extracted features, the total document evaluation score $eval(d)$ is the linear sum of feature scores:

$eval(d) = \sum_{j=1}^n score(f_j)$

The predicted class $\widehat{C}(d)$ is assigned according to the sign of $eval(d)$ :

$\widehat{C}(d) = \begin{cases} C & \text{if } eval(d) > 0 \\ C' & \text{if } eval(d) < 0 \end{cases}$
Knowl 2 — Cross-Category and In-Domain Evaluation Setup for Review Sentiment Classification
experimental setup
Experiments evaluate sentiment classification on user reviews spidered from C|net across seven consumer electronics categories, where ground truth is provided by binary user tags (thumbs-up for positive, thumbs-down for negative):
- Networking kits: 13 products, 191 positive, 144 negative
- TVs: 143 products, 743 positive, 119 negative
- Laser printers: 74 products, 1,088 positive, 439 negative
- Cheap laptops: 147 products, 3,057 positive, 683 negative
- PDAs: 83 products, 3,335 positive, 896 negative
- MP3 players: 118 products, 5,418 positive, 2,108 negative
- Digital cameras: 173 products, 12,078 positive, 1,275 negative
Two distinct testing protocols evaluate different generalization dimensions:
1. Test 1 (Cross-Category Generalization with Natural Skew): Leave-one-category-out validation where the model is trained on 6 categories and tested on the remaining 7th category, repeating for all 7 categories and reporting the macroaverage accuracy. This retains the natural 5:1 positive skew, duplicate posts, and short reviews (20% have fewer than 10 tokens).
2. Test 2 (Balanced In-Domain Classification): 10 independent trials sampled from the 4 largest categories (Cheap laptops, PDAs, MP3 players, Digital cameras). Each trial uses 10 balanced subsets of 56 positive and 56 negative reviews per category (448 deduplicated reviews with $> 10$ tokens each per set), training on 9 subsets and testing on the remaining 1, reporting the average accuracy over the 10 runs.

Knowl 3 — Comparative Performance of Information Retrieval Scoring and Machine Learning Classifiers

empirical result

Classification accuracy of the baseline information retrieval (IR) bias scoring method was evaluated against standard machine learning classifiers—Support Vector Machines ( $SVM^{light}$ ), Naive Bayes with various smoothing methods, Maximum Entropy, and Expectation Maximization—as well as alternative IR scoring functions across Test 1 (cross-category with skew) and Test 2 (balanced in-domain):

Method / Scoring Scheme	Test 1 Accuracy		Test 2 Accuracy
	Unigrams	Bigrams	Unigrams	Bigrams
Baseline IR Bias Score	85.0%	88.3%	82.2%	84.6%
SVM	81.1%	87.2%	84.4%	85.8%
Naive Bayes (Unsmoothed)	77.0%	–	80.1%	–
Naive Bayes (Laplace smoothing)	87.0%	86.9%	80.1%	81.9%
Naive Bayes (Witten-Bell)	83.1%	–	80.3%	–
Naive Bayes (Good-Turing)	76.8%	–	80.1%	–
Maximum Entropy	–	–	82.0%	–
Expectation Maximization	–	–	81.2%	–
Odds Ratio (Presence model)	53.3%	–	83.3%	–
Odds Ratio (Term probability)	84.7%	–	82.6%	85.4%
Probabilities after Thresholding	76.3%	–	82.7%	–
Information Gain Scoring	81.6%	–	80.6%	–
Fisher Discriminant	76.3%	–	56.9%	–
All Positive Baseline	76.3%	–	50.0%	–

The baseline IR bigram scoring achieves $88.3\%$ accuracy on Test 1, exceeding SVM ( $87.2\%$ ). On Test 2, the SVM bigram score ( $85.8\%$ ) is statistically indistinguishable ( $t=0.527$ ) from the best IR variant, the Odds Ratio bigram score ( $85.4\%$ ). Parametric discriminants such as the Fisher Discriminant suffer severe degradation ( $56.9\%$ on Test 2) in noisy review text.

Knowl 4 — Impact of N-gram Higher Orders and Token Substitution on Review Classification

empirical result

The effect of higher-order n-grams and entity substitutions on classification accuracy was measured using the baseline IR bias scoring metric on C|net reviews:

Feature / Substitution	Bigrams		Trigrams
	Test 1	Test 2	Test 1	Test 2
Baseline (No substitutions)	88.3%	84.6%	88.7%	84.5%
NUMBER substitution	88.3%	84.7%	88.4%	84.2%
Product Name substitution	88.3%	–	88.8%	84.2%
NUMBER + Product Name	88.2%	84.7%	88.9%	84.6%
Global (Rare word) substitution	88.3%	–	88.7%	–

Key findings include:

Higher-order n-grams outperform unigram baselines ( $85.0\%$ on Test 1, $82.2\%$ on Test 2), with trigrams reaching peak performance ( $88.7\%$ ) on Test 1 and bigrams reaching $84.6\%$ on Test 2.
Mixing lower-order features (e.g., unigrams combined with bigrams) degrades overall accuracy unless lower-order features are weighted down to $\le 25\%$ of higher-order feature weights.
Substituting numerical tokens with a generic NUMBER token prevents misleading positive/negative associations (such as isolated numbers like "64" inheriting positive weight from "64 MB").
Substituting product name occurrences with productname enables generalizing subjective phrases across entities (e.g., "I called Nikon" becomes "I called productname").
Replacing low-frequency words globally with a unique token degrades accuracy due to overgeneralization.

Knowl 5 — Performance Impact of Linguistic Features and Syntactic Preprocessing

empirical result

Incorporating linguistic parsers, semantic taxonomies, morphological stemming, and explicit negation marking was evaluated against the unigram baseline ( $84.9\%$ on Test 1, $82.2\%$ on Test 2):

Linguistic Feature / Transformation	Test 1 Accuracy	Test 2 Accuracy
Unigram Baseline	84.9%	82.2%
WordNet Synset Expansion	81.5%	80.2%
MINIPAR Syntactic Colocations	83.3%	77.3%
Porter Stemming	84.5%	83.0% ( $t=3.787$ )
Explicit Negation Tagging	81.9%	81.5%

Key observations:

WordNet: Mapping words uniformly to all synsets without word sense disambiguation introduces substantial noise (e.g., associating "duds" and "threads" with clothing in electronics reviews), degrading Test 1 accuracy by $3.4$ percentage points.
MINIPAR Colocations: Representing relationships as syntactic triplets (e.g., Word(POS):Relation:Word(POS)) significantly underperforms surface n-grams ( $77.3\%$ on Test 2).
Porter Stemming: Stemming provides a statistically significant improvement on in-domain Test 2 ( $83.0\%$ , $t=3.787$ ), but degrades cross-domain performance on Test 1 ( $84.5\%$ ) because morphological details like tense carry sentiment information (e.g., past tense verbs frequently signal product returns in negative reviews).
Negation Tagging: Heuristically prepending NOT to all words following "not" or "never" decreases accuracy; contiguous n-grams capture negation phrases more reliably.

Knowl 6 — Variable-Length Substring Feature Selection via Suffix Tree Pruning
algorithm
An algorithm for identifying arbitrary-length substring features from review text by traversing a suffix tree and pruning descendant nodes whose evidence-differentiation tradeoff does not exceed that of their parent:
Input: Corpus of tokenized training documents $D$ , information gain threshold $\theta$ , class set $\{C, C'\}$
Output: Scored substring feature dictionary $F$
Build a suffix tree (or suffix array) $T$ over all token sequences in $D$ up to a maximum length cutoff
Initialize queue $Q$ with root node of $T$
while $Q$ is not empty do
    Pop node $u$ from $Q$
    for each child node $v$ of $u$ in $T$ do
        Compute $IG(v \mid u)$ , the information gain of child substring $v$ relative to parent $u$
        if $IG(v \mid u) \ge \theta$ then
            Compute document frequency $df(v)$ of substring $v$ in $D$
            Compute class probabilities $p(C \mid v)$ and $p(C' \mid v)$
            Compute intensity $int(v) = |p(C \mid v) - p(C' \mid v)|$
            Assign feature score $score(v) = int(v) \cdot df(v)$
            Add $(v, score(v))$ to feature dictionary $F$
            Push $v$ onto $Q$
        end if
    end for
end while
return $F$
At test time, documents are classified by extracting the longest matching substring features starting at each token position and summing their scores. When scored using $int \cdot df$ , this representation achieves $85.1\%$ accuracy on Test 2 (compared to $84.5\%$ for fixed trigrams) and reaches $85.3\%$ when paired with product name substitution.
Knowl 7 — Web Review Mining and Sentence-Level Filtering Pipeline
model/method
The end-to-end pipeline for mining product opinions from unconstrained web search results consists of the following sequential stages:
1. Search Engine Crawling: Issue search queries for a specific product name and crawl the resulting web pages.
2. Heuristic Document & Paragraph Filtering:
  - Page filter: Discard pages that do not contain the term "review" in the HTML <title> tag.
  - Paragraph filter: Discard paragraphs that do not explicitly contain the target product name.
  - Length filter: Discard excessively short or excessively long sentences to eliminate headers, navigation links, and run-on boilerplate.
3. Sentence Extraction & Tokenization: Strip HTML tags, segment text into individual sentences, and tokenize each sentence.
4. Sentence-Level Scoring: Score each extracted sentence $s$ using the classifier trained on structured reviews, computing polarity $eval(s) = \sum_j score(f_j)$ and confidence magnitude $|eval(s)|$ .

Knowl 8 — Sentence Sentiment Accuracy Stratified by Classifier Confidence

empirical result

Sentence-level sentiment classification was evaluated on 600 candidate sentences mined from web search results across 3 products (200 sentences per product). Manual tagging categorized sentences into positive ( $P$ , 173 sentences), negative ( $N$ , 71 sentences), or indeterminate/irrelevant/non-opinion ( $I$ , 356 sentences, accounting for $59.3\%$ of all mined sentences).

When non-opinion and ambiguous sentences ( $I$ ) are removed, the classifier's confidence score correlates directly with accuracy: the top-confidence tercile achieves $76\%$ binary accuracy ( $P$ vs $N$ ), compared to $58\%$ for the middle tercile and $34\%$ for the lowest tercile. Dynamic programming substring scoring achieves $68\%$ overall binary accuracy on mined sentences, outperforming baseline bigrams/trigrams ( $61\%\text{--}62\%$ ).

Knowl 9 — Product Attribute Extraction via Definite Article Bigram Grouping
model/method
To organize mined review sentences into structured product summaries, product attributes are extracted and grouped using definite article bigram patterns:
1. Candidate Attribute Extraction: Extract bigrams conforming to the pattern "the" + [word] from the mined sentence set (e.g., "the screen", "the memory", "the palmOS" for PDAs; "the taste", "the flavor", "the calories" for beer).
2. Frequency and Stopword Thresholding: Filter extracted bigrams using stopword lists and minimum frequency thresholds.
3. Sentence Clustering: Group all review sentences containing the specific attribute bigram under its respective attribute heading.
4. Attribute-Level Sentiment Aggregation: Compute an aggregate sentiment score for each attribute by averaging the sentiment scores of all sentences grouped under it, displaying individual feature contributions and context in the user interface.
This simple syntactic pattern extracts cleaner, more descriptive product attributes than category-specific dictionary selection or mutual information clustering.
Knowl 10 — Structural Challenges in Automated Review Opinion Mining
limitation
Four systemic characteristics of web product reviews impede automated sentiment extraction and classification:
1. Rating Inconsistency: Discrepancies between qualitative textual sentiment and quantitative star ratings, including user errors where positive text is assigned a 1-star rating by mistake.
2. Ambivalence and Comparative Sentiment: Mixed reviews containing ambivalent remarks (e.g., negative expressions followed by an equivocating concluding endorsement) or cross-product comparisons (contrasting a positive experience with one brand against a poor experience with another), introducing noisy associations between negative phrases and overall positive ratings.
3. Vocabulary Sparsity and Review Length Asymmetry: User reviews are frequently short, requiring classifiers to capture highly specific features. In the C|net corpus, over two-thirds of distinct vocabulary words appear in fewer than 3 documents.
4. Class and Product Skew: Severe natural class imbalance, with positive reviews outnumbering negative reviews by approximately 5 to 1, and popular product categories dominating corpus statistics. This causes common entity terms (such as "camera") to artificially acquire positive sentiment scores.

Coverage note — Preliminary exploratory experiments on scalar 5-star regression on the Amazon corpus were omitted because they were described as preliminary trials that clustered predictions at the mean and did not alter the main binary classification framework.

References

1.Sanjiv Ranjan Das and Mike Y. Chen. Yahoo! for Amazon: Sentiment parsing from small talk on the web. In Proceedings of the 8th Asia Pacific Finance Association Annual Conference, 2001.
2.Fernando Pereira et al. Beyond word N-grams. In David Yarovsky and Kenneth Church, editors, Proceedings of the Third Workshop on Very Large Corpora, pages 95–106, Somerset, New Jersey, 1995. Association for Computational Linguistics.
3.Aidan Finn, Nicholas Kushmerick, and Barry Smyth. Genre classification and domain transfer for information filtering. In Fabio Crestani, Mark Girolami, and Cornelis J. van Rijsbergen, editors, Proceedings of ECIR-02, 24th European Colloquium on Information Retrieval Research, Glasgow, UK. Springer Verlag, Heidelberg, DE.
4.W. Gale. Good-Turing smoothing without tears. Journal of Quantitative Linguistics, 2:217–37, 1995.
5.Vasileios Hatzivassiloglou and Kathleen R. McKeown. Predicting the semantic orientation of adjectives. In Proceedings of the 35th Annual Meeting of ACL, 1997.
6.Vasileios Hatzivassiloglou and Janyce M. Wiebe. Effects of adjective orientation and gradability on sentence subjectivity. In Proceedings of the 18th International Conference on Computational Linguistics, 2000.
7.M. Hearst. Direction-Based Text Interpretation as an Information Access Refinement. 1992.
8.David Holtzmann. Detecting and tracking opinions in on-line discussions. In UCB/SIMS Web Mining Workshop, 2001.
9.Dekang Lin. Automatic retrieval and clustering of similar words. In Proceedings of COLING-ACL, pages 768–774, 1998.
10.Hugo Liu, Henry Lieberman, and Ted Selker. A model of textual affect sensing using real-world knowledge. In Proceedings of the Seventh International Conference on Intelligent User Interfaces, pages 125–132, 2003.
11.Andrew Kachites McCallum. Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. http://www.cs.cmu.edu/ mccallum/bow, 1996.
12.Dunja Mladenic. Feature subset selection in text-learning. In European Conference on Machine Learning, pages 95–100, 1998.
13.R. Mooney, P. Bennett, and L. Roy. Book recommending using text categorization with extracted information. In Proceedings of the AAAI Workshop on Recommender Systems, 1998.
14.Satoshi Morinaga, Kenji Yamanishi, Kenji Tateishi, and Toshikazu Fukushima. Mining product reputions on the web. In KDD 2002.
15.Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 79–86.
16.Fernando C. N. Pereira, Naftali Tishby, and Lillian Lee. Distributional clustering of English words. In Meeting of the Association for Computational Linguistics, pages 183–190, 1993.
17.M.F. Porter. An algorithm for suffix stripping. In Program, volume 14, pages 130–137, 1980.
18.Deepak Ravichandran and Eduard Hovy. Learning surface text patterns for a question answering system. In ACL Conference, 2002.
19.Ellen Riloff. Automatically generating extraction patterns from untagged text. In Proceedings of AAAI/IAAI, Vol. 2, pages 1044–1049, 1996.
20.P. Subasic and A. Huettner. Affect analysis of text using fuzzy semantic typing. IEEE-FS, 9:483–496, Aug. 2001.
21.Loren Terveen, Will Hill, Brian Amento, David McDonald, and Josh Creter. PHOAKS: A system for sharing recommendations. Communications of the ACM, 40(3):59–62, 1997.
22.Richard M. Tong. An operational system for detecting and tracking opinions in on-line discussion. In SIGIR Workshop on Operational Text Classifiation, 2001.
23.P.D. Turney and M.L. Littman. Unsupervised learning of semantic orientation from a hundred-billion-word corpus. Technical Report ERB-1094, National Research Council Canada, Institute for Information Technology, 2002.
24.Janyce Wiebe. Learning subjective adjectives from corpora. In AAAI/IAAI, pages 735–740, 2000.
25.Janyce Wiebe, Rebecca Bruce, Matthew Bell, Melanie Martin, and Theresa Wilson. A corpus study of evaluative and speculative language. In Proceedings of the 2nd ACL SIGdial Workshop on Discourse and Dialogue, 2001.
26.Janyce Wiebe, Theresa Wilson, and Matthew Bell. Identifying collocations for recognizing opinions. In Proceedings of ACL/EACL 2001 Workshop on Collocation.
27.Mikio Yamamoto and Kenneth Church. Using suffix arrays to compute term frequency and document frequency for all substrings in a corpus. In Proceedings of the 6th Workshop on Very Large Corpora.

Citation

MLA

Dave, K., et al. “Mining the Peanut Gallery”. Proceedings of the Twelfth International Conference on World Wide Web - WWW '03, 2003, p. 519, https://doi.org/10.1145/775152.775226.

APA

Dave, K., Lawrence, S., & Pennock, D. M. (2003). Mining the peanut gallery. Proceedings of the Twelfth International Conference on World Wide Web - WWW '03, 519. https://doi.org/10.1145/775152.775226

Chicago

Dave, K., S. Lawrence, and D. M. Pennock. 2003. “Mining the Peanut Gallery”. Proceedings of the Twelfth International Conference on World Wide Web - WWW '03, 519. https://doi.org/10.1145/775152.775226.

Harvard

Dave, K., Lawrence, S. and Pennock, D.M. (2003) “Mining the peanut gallery”, Proceedings of the twelfth international conference on World Wide Web - WWW '03. ACM Press, p. 519. Available at: https://doi.org/10.1145/775152.775226.

Vancouver

1. Dave K, Lawrence S, Pennock DM (2003) Mining the peanut gallery. In: Proceedings of the twelfth international conference on World Wide Web - WWW '03. ACM Press, p 519

BibTeX

@inproceedings{Dave_2003, series={WWW ’03}, title={Mining the peanut gallery: opinion extraction and semantic classification of product reviews}, url={http://dx.doi.org/10.1145/775152.775226}, DOI={10.1145/775152.775226}, booktitle={Proceedings of the twelfth international conference on World Wide Web  - WWW ’03}, publisher={ACM Press}, author={Dave, Kushal and Lawrence, Steve and Pennock, David M.}, year={2003}, pages={519}, collection={WWW ’03} }

Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Confidence Group (Tercile)	Raw 3-Class Accuracy	Accuracy Excluding Indeterminate ( $I$ )
Top 200 (Highest Confidence)	42%	76%
Middle 200	21%	58%
Lowest 200 (Lowest Confidence)	50%	34%

Mining the peanut gallery: opinion extraction and semantic classification of product reviews

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Normalized Frequency Bias Scoring and Classification Rule

Knowl 2 — Cross-Category and In-Domain Evaluation Setup for Review Sentiment Classification

Knowl 3 — Comparative Performance of Information Retrieval Scoring and Machine Learning Classifiers

Knowl 4 — Impact of N-gram Higher Orders and Token Substitution on Review Classification

Knowl 5 — Performance Impact of Linguistic Features and Syntactic Preprocessing

Knowl 6 — Variable-Length Substring Feature Selection via Suffix Tree Pruning

Knowl 7 — Web Review Mining and Sentence-Level Filtering Pipeline

Knowl 8 — Sentence Sentiment Accuracy Stratified by Classifier Confidence

Knowl 9 — Product Attribute Extraction via Definite Article Bigram Grouping

Knowl 10 — Structural Challenges in Automated Review Opinion Mining

References

Citation

BibTeX

Access the Paper