Stance Detection on Social Media with Background Knowledge
Ang LiBin LiangJingqian ZhaoBowen ZhangMin YangRuifeng Xu
Proposes a knowledge-augmented stance detection framework that integrates Wikipedia-derived episodic context and language-model-interpreted social discourse to achieve state-of-the-art accuracy in both in-target and zero-shot settings.
Social media stance detection aims to automatically determine whether an author supports, opposes, or remains neutral toward a specific target or topic. However, social media posts are typically brief and heavily laden with slang, acronyms, and hashtags, while omitting the essential factual background required to comprehend implicit viewpoints. Existing automated methods struggle because they rely primarily on surface-level text or lack domain-general background information, making accurate public opinion analysis difficult.
The article evaluates whether augmenting stance detection models with external background knowledge improves classification performance. Specifically, it demonstrates the Knowledge-Augmented Stance Detection framework, which categorizes background context into factual understanding of the topic (episodic knowledge) and interpretations of social media informal language and references (discourse knowledge).
To implement this approach, the authors developed a heuristic retrieval pipeline that crawls Wikipedia pages relevant to specific topics and uses topic modeling alongside text-matching algorithms to identify relevant sections. A large language model filters this retrieved content to remove non-essential text and paraphrase informal social media posts by resolving abbreviations and hashtags. The framework was evaluated across four standard benchmark datasets covering political figures, COVID-19 policies, and diverse debate topics under both known-target (in-target) and unseen-target (zero-shot) conditions.
The analysis yields four critical findings. First, the framework consistently achieves top performance across standard benchmarks, significantly outperforming existing baseline models and pre-trained architectures. Second, knowledge augmentation substantially improves zero-shot stance detection on unseen topics, raising average performance on challenging datasets from roughly 40–53% up to 58%. Third, human evaluations confirmed high data extraction fidelity, with independent reviewers rating the generated and filtered knowledge as accurate and relevant in over 95% of sampled cases. Fourth, distilling background and contextual knowledge into smaller, specialized language models enables them to outperform large general-purpose foundation models while operating with 500 to 1,000 times fewer parameters.
These findings indicate that smaller, fine-tuned models augmented with targeted external knowledge provide a superior, cost-effective alternative to relying entirely on massive, computationally expensive language models for public opinion monitoring. Organizations can deploy compact, highly accurate stance detection systems that lower computational overhead, reduce deployment latency, and mitigate the risk of classification errors caused by ambiguous online slang.
Decision-makers building opinion monitoring or social media analytics capabilities should consider adopting knowledge-retrieval frameworks before investing heavily in larger proprietary models. Implementing a hybrid approach—using larger language models offline to retrieve and filter knowledge bases while running compact fine-tuned models in production—offers the best balance of accuracy and operating cost. Further work should explore incorporating real-time news sources to address emerging topics beyond static reference materials.
The primary limitation of this study is its reliance on pre-crawled Wikipedia articles, which may limit effectiveness for rapidly evolving, breaking events where online encyclopedias are not immediately updated. Nevertheless, given the consistent empirical improvements across diverse datasets and high human-evaluation scores, there is high confidence in the framework's effectiveness for general and zero-shot stance detection tasks.
- Paper: JointCL: A Joint Contrastive Learning Framework for Zero-Shot Stance Detection, Bin Liang et al. (2022). JointCL establishes the zero-shot stance-detection problem and transfer setting that KASD uses to test whether retrieved background knowledge improves generalization to unseen targets.
- Paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Patrick Lewis et al. (2020). RAG introduces retrieving external Wikipedia passages to augment a model’s input, providing the core retrieval-and-knowledge-integration approach that KASD adapts for stance detection.
- Paper: Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification, Shengding Hu et al. (2022). KPT shows how external knowledge can enrich text-classification decisions, a key precedent for KASD’s use of retrieved knowledge to resolve missing context.
- Paper: Supervised Topic Models, David M. Blei et al. (2007). Supervised topic models provide the topic-modeling foundation that helps explain KASD’s use of topic structure to identify relevant background documents.
No sufficiently relevant recommendations were found.
