Identifying the Human Values behind Arguments
Johannes KieselMilad AlshomaryNicolas HandkeXiaoni CaiHenning WachsmuthBenno Stein
Presents a psychology-grounded taxonomy of 54 human values alongside a cross-cultural dataset of 5,270 annotated arguments to establish computational baselines for identifying the underlying ethical principles in natural language reasoning.
When people disagree on public policy or controversial issues, their differing stances often stem from conflicting human values—such as personal achievement versus tradition—rather than access to different facts. While human values provide the core persuasive foundation for arguments, natural language processing and argument analysis tools have struggled to evaluate them automatically because values are vast in variety, abstract, and frequently left implicit in text.
The article aims to define and demonstrate the first computational approach for automatically identifying the human values behind written natural language arguments across multiple levels of granularity and diverse geographic regions.
To achieve this, the article establishes a consolidated multi-level taxonomy of 54 values structured into four hierarchy tiers, building on established social science frameworks. The researchers then compiled a benchmark dataset of 5,270 arguments sourced from four regions: the United States (5,020 arguments), Africa (50), China (100), and India (100). Three crowdworkers annotated each argument across all 54 values, yielding roughly 850,000 individual judgments. The dataset was then evaluated using baseline classification models, support vector machines, and fine-tuned pre-trained language models based on bidirectional encoder representations.
The findings show that natural language models can identify specific underlying human values, outperforming baseline models on fine-grained categories with an average macro F1-score of 0.25 and reaching up to 0.81 for prominent values such as personal health. Higher-level value categories, however, proved more challenging to separate cleanly, as arguments routinely appeal to multiple broader dimensions simultaneously. Performance generally tracked label frequency, showing stronger detection for well-represented values. Furthermore, models trained predominantly on United States data demonstrated robust cross-cultural transferability, achieving comparable or superior performance on the African, Chinese, and Indian datasets.
These results demonstrate that computational value identification is feasible, providing a foundation for systems that can automatically detect why audiences accept specific claims. In practical applications, this capability can support audience-aware communication, improve automated argument quality assessments, and help bridge polarized debates by identifying shared underlying values between opposing viewpoints. However, low average precision indicates that standard language models cannot yet be deployed autonomously without substantial human review.
Organizations and researchers pursuing value-based automated reasoning should invest in hierarchical multi-label classification architectures and expand training data for less frequent values. Future initiatives must prioritize broader international datasets and recruit culturally diverse annotators to mitigate regional bias.
Confidence in these findings should be weighed against notable limitations, including an extreme class imbalance where over 95 percent of the dataset originates from the United States, as well as the reliance on Western-based crowdworkers to interpret arguments across multiple cultural contexts.
No sufficiently relevant recommendations were found.
No sufficiently relevant recommendations were found.
