A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs

Finn Årup Nielsen

article2011#MSM1,398 citations

Presents a sentiment lexicon specifically designed for microblogs, demonstrating that adapting affective word lists to informal Twitter text improves valence scoring compared to traditional resources like ANEW.

Listen

Analyzing sentiment on microblogging platforms like Twitter has become increasingly vital for understanding public mood, brand perception, and real-time social trends. However, conventional sentiment analysis lexicons—such as the Affective Norms for English Words (ANEW)—were created prior to the widespread adoption of microblogging and lack the informal slang, acronyms, and vulgarities typical of online communication. This raises questions about how accurately traditional word lists measure emotion in short, informal texts.

The article evaluates whether a newly constructed sentiment lexicon tailored specifically for microblogs outperforms established word lists in estimating the sentiment strength of Twitter posts. Specifically, it compares the author's 2,477-word list against ANEW, General Inquirer, OpinionFinder, and the specialized SentiStrength software.

To conduct the assessment, the author scored the new lexicon manually on a scale from minus five to plus five, incorporating internet slang and obscenities. The performance of all approaches was benchmarked against an independent ground-truth dataset of 1,000 tweets, each evaluated ten times by human annotators via Amazon Mechanical Turk. Lexicon performance was primarily measured by calculating the correlation between automated sentiment scores and the human benchmark ratings.

The evaluation revealed several key findings. First, the specialized SentiStrength software achieved the highest accuracy, with a correlation of 0.610. Second, among simple word-matching methods, the author's new lexicon outperformed ANEW, achieving a correlation of 0.564 compared to ANEW's 0.525. Third, polarity-only lexicons such as OpinionFinder and General Inquirer performed significantly worse, posting correlations of only 0.458 and 0.374, respectively. Finally, a direct comparison using only the overlapping words between the new list and ANEW showed that ANEW's ratings were marginally superior (0.52 versus 0.49), indicating that the advantage of the new lexicon stems from its broader microblog-focused vocabulary rather than better valence calibration.

These findings demonstrate that while sophisticated algorithmic tools like SentiStrength provide the highest accuracy, lexicon-based methods can achieve competitive results if they include platform-specific language, such as online acronyms and informal expressions. Furthermore, assigning graded sentiment strength rather than simple positive or negative classifications is essential for informal text analysis. Organizations seeking low-complexity, real-time sentiment tracking can use tailored lexicons as a cost-effective alternative to complex software pipelines without substantial performance loss.

Teams deploying sentiment analysis on social data should prioritize lexicons that incorporate internet slang and support continuous vocabulary expansion. When maximum precision is required, organizations should favor multi-rule tools like SentiStrength that account for spelling variations and grammatical context. Further work should explore whether adding automated negation handling and emoticon detection improves simple word lists without introducing undue computational complexity.

Confidence in these results is moderate to high for basic Twitter text, supported by human consensus ratings across 1,000 posts. However, users should remain cautious because the new word list relied on subjective ratings from a single annotator, and the benchmark dataset was limited to a single 1,000-tweet sample, which may not capture all regional dialects or emerging online jargon.

Technical University of Denmark micro-blog sentiment analysis affective state sentiment analysis sentiment lexicons Affective Norms for English Words

arXiv: 1103.2903 fnielsen/afinn

Paper: Measuring praise and criticism: Inference of semantic orientation from association, Peter D. Turney et al. (2003). This seminal paper introduces methods for inferring the semantic orientation and sentiment valence of individual words, establishing foundational concepts evaluated in microblog lexicons.
Paper: Thumbs up? Sentiment Classification using Machine Learning Techniques, Bo Pang et al. (2002). This benchmark work formalizes computational sentiment analysis and baseline word-matching strategies that motivate the need for specialized microblog lexicons.
Paper: Predicting the Semantic Orientation of Adjectives, Vasileios Hatzivassiloglou et al. (1997). It provides the foundational framework for determining the semantic orientation and polarity of descriptive words used across affective word lists.
Paper: A holistic lexicon-based approach to opinion mining, Xiaowen Ding et al. (2008). It establishes core lexicon-based sentiment analysis methodologies and word scoring techniques directly compared and evaluated in microblog contexts.
Paper: Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews, Peter D. Turney (2002). This paper demonstrates how unsupervised semantic orientation of words can be aggregated to determine overall text sentiment, a direct precursor to lexicon-based tweet evaluation.
Paper: What is Twitter, a social network or a news media?, Haewoon Kwak et al. (2010). It details the core linguistic and structural characteristics of Twitter communication that make standard pre-microblogging lexicons like ANEW less effective.
Paper: Twitter mood predicts the stock market, Johan Bollen et al. (2010). It illustrates early applications and limitations of lexicon-based sentiment tracking over microblog streams like Twitter.

Paper: CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON, Saif M. Mohammad et al. (2013). It advances beyond simple microblog valence lists by using crowdsourcing to construct a comprehensive term-emotion and polarity association lexicon.
Paper: Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, Richard Socher et al. (2013). It moves past bag-of-words lexicon matching by modeling syntactic compositionality and phrase-level sentiment over treebanks.
Paper: Convolutional Neural Networks for Sentence Classification, Yoon Kim (2014). It extends short-text sentiment classification from simple word-list matching to deep convolutional neural networks built on distributed word embeddings.
Paper: Deep learning for sentiment analysis: A survey, Lei Zhang et al. (2018). It offers a comprehensive survey detailing the progression from rule- and lexicon-based microblog analysis to modern deep neural sentiment architectures.
Paper: Automated Hate Speech Detection and the Problem of Offensive Language, Thomas Davidson et al. (2017). It applies Twitter lexical and sentiment features to fine-grained downstream classification tasks distinguishing offensive language from hate speech.

Cover for A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs

Abstract

Sentiment analysis of microblogs such as Twitter has recently gained a fair amount of attention. One of the simplest sentiment analysis approaches compares the words of a posting against a labeled word list, where each word has been scored for valence, -- a 'sentiment lexicon' or 'affective word lists'. There exist several affective word lists, e.g., ANEW (Affective Norms for English Words) developed before the advent of microblogging and sentiment analysis. I wanted to examine how well ANEW and other word lists performs for the detection of sentiment strength in microblog posts in comparison with a new word list specifically constructed for microblogs. I used manually labeled postings from Twitter scored for sentiment. Using a simple word matching I show that the new word list may perform better than ANEW, though not as good as the more elaborate approach found in SentiStrength.

Topics

micro-blog sentiment analysis affective state part of speech

Keywords

sentiment analysis sentiment lexicons Affective Norms for English Words affective word lists sentiment strength detection microblog sentiment SentiStrength algorithm Amazon Mechanical Turk General Inquirer

1 Introduction
2 Construction of word list
3 Twitter data
4 Results
5 Discussion
References

Knowls

Knowl 1 — AFINN Sentiment Lexicon Construction and Scoring Design
model/method
The AFINN sentiment lexicon is an affective word list specifically constructed for sentiment strength detection in microblogs and short informal text (such as Twitter). Each word or short phrase is scored manually for valence on an integer scale from $-5$ (very negative) to $+5$ (very positive).

Lexicon characteristics and design choices include:
- Vocabulary scale and expansion: An initial version (termed AFINN-96) contains 1,468 words. The expanded version comprises 2,477 unique entries (including 15 phrases).
- Sources: The lexicon was initiated from sets of obscene words and positive seed words, and expanded using Twitter postings related to the COP15 climate conference, the Original Balanced Affective Word List by Greg Siegle, Steven J. DeRose's The Compass DeRose Guide to Emotion Words, Wiktionary synonym entries, Microsoft Web n-gram context similarity clusters, and Urban Dictionary slang and acronyms (e.g., WTF, LOL, ROFL).
- Disambiguation and filtering: To prevent part-of-speech ambiguity in simple matching, words with variable polarity or category shifts (e.g., patient, firm, mean, power, frank) were excluded. High-arousal words with context-dependent sentiment (e.g., surprise) were also omitted.
- Valence distribution: The lexicon exhibits an asymmetric bias toward negative terms, containing 1,598 negative entries (65%), 878 positive entries (35%), and 1 neutral phrase. The modal positive rating is $+2$ , the modal negative rating is $-2$ , and strong obscenities are assigned $-4$ or $-5$ .

Knowl 2 — Comparative Sentiment Strength Correlation on Microblog Posts

data/table

Evaluation of sentiment strength detection methods on a benchmark corpus of 1,000 English tweets. Each tweet was independently scored by 10 Amazon Mechanical Turk (AMT) raters on an integer scale from 1 (negative) to 9 (positive), with the mean score defining the ground truth.

Five methods were compared: AFINN (2,477 words, valence $[-5, +5]$ ), ANEW (1,034 words, valence $[1, 9]$ ), General Inquirer (GI, binary polarity mapped to $\{-1, +1\}$ ), OpinionFinder (OF, binary polarity mapped to $\{-1, +1\}$ ), and SentiStrength (SS, rule-based microblog analyzer).

Method	AMT	AFINN (My)	ANEW	GI	OF
AFINN (My)	.564
ANEW	.525	.696
GI	.374	.525	.592
OF	.458	.675	.624	.705
SS	.610	.604	.546	.474	.512

Spearman rank correlations against AMT ratings are: AFINN ( $\rho = 0.596$ ), ANEW ( $\rho = 0.544$ ), GI ( $\rho = 0.422$ ), OF ( $\rho = 0.491$ ), and SentiStrength ( $\rho = 0.616$ ). In terms of token coverage over the 4,095 unique words identified in the 1,000 tweets, AFINN matched 422 words, ANEW matched 398 words, GI matched 358 words (out of 3,392 non-zero words), and OF matched 562 words (out of 6,442 words).

Knowl 3 — Lexicon-Based Tweet Sentiment Scoring and Aggregation Schemes
model/method
Let a tweet $T = (w_1, w_2, \dots, w_N)$ be a sequence of $N$ tokenized words, and let $v(w) \in \mathbb{R}$ denote the sentiment valence assigned to word $w$ by a sentiment lexicon $L$ (with $v(w) = 0$ if $w \notin L$ ).

The primary sentiment strength score $S(T)$ computes the mean valence across all tokens in the tweet: $S(T) = \frac{1}{N} \sum_{i=1}^{N} v(w_i)$

Alternative aggregation formulations evaluated on the 1,000-tweet benchmark dataset include:
- Unnormalized sum: $S_{\text{sum}}(T) = \sum_{i=1}^N v(w_i)$ , which yielded the highest Pearson correlation for AFINN ( $r = 0.581$ ).
- Active-token normalization: Normalizing the sum by the count of tokens that have non-zero valence: $N_{\text{active}} = |\{w_i : v(w_i) \neq 0\}|$ .
- Extreme valence selection: Selecting the single valence with the largest absolute magnitude: $\arg\max_{v(w_i)} |v(w_i)|$ , which yielded the lowest Pearson correlation for AFINN ( $r = 0.543$ ).
- Ternary quantization: Quantizing the aggregated score to $\{-1, 0, +1\}$ , which yielded $r = 0.548$ for AFINN.
For ANEW, Pearson correlation remained stable across all four scoring schemes ( $r \in [0.522, 0.526]$ ). For AFINN, normalizing by total word count $N$ achieved the highest Spearman rank correlation ( $\rho = 0.596$ ).
Knowl 4 — Vocabulary Coverage vs. Valence Scoring in AFINN and ANEW
empirical result
To isolate whether AFINN's performance advantage over ANEW on Twitter text ( $r = 0.564$ vs. $r = 0.525$ ) originates from superior valence calibration or broader vocabulary coverage, both lexicons were evaluated strictly on their intersection.

Direct exact matching between AFINN and ANEW yielded an overlapping vocabulary of 299 words. Two sentiment analyzers were constructed using only these 299 words: one using AFINN's $[-5, +5]$ integer valences and the other using ANEW's $[1, 9]$ mean valences.

When evaluated on the 1,000-tweet benchmark dataset:
- The ANEW valence scorer achieved a Pearson correlation of $r = 0.52$ .
- The AFINN valence scorer achieved a Pearson correlation of $r = 0.49$ .
Because ANEW outperforms AFINN when the lexicon is held constant, the overall superiority of AFINN on microblog data is attributable to its larger domain-appropriate vocabulary (incorporating slang and obscenities) rather than superior per-word valence scoring.
Knowl 5 — Lexicon Correlation and Valence Discrepancies Between AFINN and ANEW
empirical result
A direct comparison between the scores in AFINN (scaled $[-5, +5]$ ) and the mean valence scores in ANEW (scaled $[1, 9]$ ) on their overlapping words shows substantial overall agreement:
- Pearson correlation: $r = 0.91$
- Spearman rank correlation: $\rho = 0.81$
- Kendall tau correlation: $\tau = 0.63$
Applying Porter word stemming to both lexicons or WordNet lemmatization to AFINN does not significantly change these correlation values.

When splitting ANEW at neutral valence 5.0 and AFINN at neutral valence 0, qualitative polarity discrepancies emerge on specific terms: aggressive, mischief, ennui, hard, silly, alert, mischiefs, and noisy. Stemming introduces additional discrepancies, such as between alien and alienation, affection and affected, and profit and profiteer.
Knowl 6 — Sentiment Detection Performance as a Function of Lexicon Size
empirical result
Resampling experiments measuring sentiment strength correlation against 1,000 AMT-labeled tweets as a function of AFINN lexicon size (evaluated at 5, 100, 300, 500, 1,000, 1,500, 2,000, and 2,477 words over 50 resamples per size) demonstrate:
- Pearson correlation rises rapidly from $\approx 0.0$ at 5 words to $\approx 0.40$ at 300 words and $\approx 0.50$ at 1,000 words, before plateauing toward $0.564$ at 2,477 words.
- Spearman rank correlation follows a parallel trajectory, rising from $\approx 0.35$ at 100 words to $0.596$ at 2,477 words.
- Performance gains exhibit diminishing returns beyond 1,000 words, but continue to show positive slope up to the full 2,477-word list.
Knowl 7 — Limitations of AFINN Relative to Psychometric Norms and Rule-Based Analyzers
limitation
The AFINN lexicon and its baseline word-matching implementation have two notable limitations:
1. Single-annotator scoring: Valences were manually assigned by a single author without multi-rater averaging, inter-annotator agreement metrics, or standard deviations, whereas ANEW provides psycholinguistically validated norms across multiple human subjects. AFINN also excludes affective dimensions other than valence (such as arousal, dominance, and subjectivity/objectivity).
2. Lack of rule-based syntactic and morphological processing: Unlike specialized microblog sentiment systems such as SentiStrength (which achieved superior performance with Pearson $r = 0.610$ and Spearman $\rho = 0.616$ ), baseline AFINN lookup does not incorporate negation handling (valence flipping), emoticon detection, booster word scaling, or spelling correction/elongation normalization.

Coverage note — None was omitted; all primary contributions, lexicon definitions, experimental comparisons, intersection analyses, and stated limitations are fully covered.

References

1.Pang, B., Lee, L.: Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval 2(1-2) (2008) 1–135
2.Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., Kappas, A.: Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology 61(12) (2010) 2544–2558
3.Bradley, M.M., Lang, P.J.: Affective norms for English words (ANEW): Instruction manual and affective ratings. Technical Report C-1, The Center for Research in Psychophysiology, University of Florida (1999)
4.Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, Stroudsburg, PA, USA, Association for Computational Linguistics (2005)
5.Hansen, L.K., Arvidsson, A., Nielsen, F.Å., Colleoni, E., Etter, M.: Good friends, bad news — affect and virality in Twitter. Accepted for The 2011 International Workshop on Social Computing, Network, and Services (SocialComNet 2011) (2011)
6.Akkaya, C., Conrad, A., Wiebe, J., Mihalcea, R.: Amazon Mechanical Turk for subjectivity word sense disambiguation. In: Proceedings of the NAACL HLT 2010 Workshop on Creating, Speech and Language Data with Amazon’s Mechanical Turk, Association for Computational Linguistics (2010) 195–203
7.Baudhuin, E.S.: Obscene language and evaluative response: an empirical study. Psychological Reports 32 (1973)
8.Sapolsky, B.S., Shafer, D.M., Kaye, B.K.: Rating offensive words in three television program contexts. BEA 2008, Research Division (2008)
9.Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly, Sebastopol, California (June 2009)
10.Biever, C.: Twitter mood maps reveal emotional states of America. The New Scientist 207(2771) (July 2010) 14

Citation

MLA

Nielsen, F. Å. “A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs”. Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big Things Come in Small Packages (2011) 93-98, 2011, http://arxiv.org/abs/1103.2903v1.

APA

Nielsen, F. Å. (2011). A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big Things Come in Small Packages (2011) 93-98. http://arxiv.org/abs/1103.2903v1

Chicago

Nielsen, F. Å. 2011. “A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs”. Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big Things Come in Small Packages (2011) 93-98. http://arxiv.org/abs/1103.2903v1.

Harvard

Nielsen, F.Å. (2011) “A new ANEW: Evaluation of a word list for sentiment analysis in microblogs”, Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big things come in small packages (2011) 93-98 [Preprint]. Available at: http://arxiv.org/abs/1103.2903v1.

Vancouver

1. Nielsen FÅ (2011) A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big things come in small packages (2011) 93-98

BibTeX

@article{nielsen2011new,
  title = {A new ANEW: Evaluation of a word list for sentiment analysis in microblogs},
  author = {Nielsen, Finn Årup},
  year = {2011},
  journal = {Proceedings of the ESWC2011 Workshop on 'Making Sense of Microposts': Big things come in small packages (2011) 93-98},
  url = {http://arxiv.org/abs/1103.2903v1},
  eprint = {1103.2903}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/

A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — AFINN Sentiment Lexicon Construction and Scoring Design

Knowl 2 — Comparative Sentiment Strength Correlation on Microblog Posts

Knowl 3 — Lexicon-Based Tweet Sentiment Scoring and Aggregation Schemes

Knowl 4 — Vocabulary Coverage vs. Valence Scoring in AFINN and ANEW

Knowl 5 — Lexicon Correlation and Valence Discrepancies Between AFINN and ANEW

Knowl 6 — Sentiment Detection Performance as a Function of Lexicon Size

Knowl 7 — Limitations of AFINN Relative to Psychometric Norms and Rule-Based Analyzers

References

Citation

BibTeX

Source Code

Access the Paper