Explanation in Artificial Intelligence: Insights from the Social Sciences
Tim Miller
Synthesizes decades of findings from philosophy, cognitive science, and social psychology to define how explainable artificial intelligence should align with human cognitive biases and social expectations rather than developer intuition.
The paper reviews extensive research from philosophy, psychology, and cognitive science to improve how artificial intelligence systems generate explanations for their decisions. The resurgence of interest in explainable AI stems from evidence that many systems see limited adoption due to user distrust and ethical concerns, yet most current work relies on developers’ intuitions about what makes a good explanation rather than established findings on how people actually produce, select, and evaluate explanations.
The review surveyed more than 250 publications across social science fields, selecting key works for their relevance to everyday explanations of specific events rather than general scientific theories. It maps these findings onto the challenges of building explanatory agents that can justify decisions in applications such as autonomous systems, medical decision support, and machine learning models.
Four findings stand out. First, explanations are contrastive: people seek reasons why one outcome occurred rather than another expected outcome, and they judge explanations by how well they distinguish the two cases. Second, people select only a small subset of causes as the explanation, guided by cognitive biases such as abnormality, intentionality, and relevance to the contrast case. Third, while truth and likelihood matter, citing probabilities or statistical associations alone rarely satisfies users; causal accounts are preferred. Fourth, explanation is a social process shaped by the explainer’s beliefs about the recipient’s knowledge and goals, often occurring within a conversational exchange.
These results imply that many existing XAI techniques, which focus on surfacing full causal chains or feature importance scores, may fail to build genuine trust because they ignore how humans process and value explanations. Systems that present complete or probabilistic information risk overwhelming users or appearing unhelpful, while contrastive and selective explanations could reduce cognitive load and improve perceived relevance.
Designers of explanatory agents should therefore incorporate mechanisms to infer or elicit a user’s contrast case, prioritize abnormal or intentional causes, emphasize causal stories over statistics, and model the interaction as a dialogue that tracks shared knowledge. Further empirical studies are needed to test these principles in deployed AI systems and to develop practical methods for inferring contrast cases when they are not stated explicitly. The review itself is limited to existing social-science evidence and does not include new experiments with AI users, so its recommendations remain provisional until validated in technical settings.
- Paper: Towards A Rigorous Science of Interpretable Machine Learning, Finale Doshi-Velez et al. (2017). It provides a foundational taxonomy of interpretability goals and evaluation tiers that directly informs the source paper's discussion of human-centric evaluation.
- Paper: The Mythos of Model Interpretability, Zachary C. Lipton (2016). It unpacks the ambiguous terminology and motivations surrounding model interpretability, preparing the reader for the source paper's critique of intuition-driven XAI.
- Paper: A Survey of Methods for Explaining Black Box Models, Riccardo Guidotti et al. (2018). It delivers a comprehensive classification of post-hoc explanation methods and black-box models, serving as essential context for the source paper's review of explanation techniques.
- Paper: Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI, Alejandro Barredo Arrieta et al. (2020). It offers an extensive literature review on explainability concepts and taxonomies that complements the source paper's focus on bridging artificial intelligence with social science insights.
- Paper: From Local Explanations to Global Understanding with Explainable AI for Trees, Scott M. Lundberg et al. (2020). It advances the practical application of game-theoretic explanations by scaling exact Shapley values from local instances to global model understanding.
- Paper: Shortcut learning in deep neural networks, Robert Geirhos et al. (2020). It investigates shortcut learning in deep networks, extending the source paper's concerns about opacity and unverified model reasoning into failure analyses.
