Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Ben Shneiderman
Presents a framework for pairing high automation with high human control, guiding designers to build reliable, safe, and trustworthy AI systems that preserve human responsibility and agency.
Artificial intelligence development has long operated on the assumption that increasing machine autonomy requires reducing human control. This traditional view has led to system designs that obscure operations from users, fail in unexpected environments, and cause fatal accidents in safety-critical domains such as aviation and transportation.
The article establishes a two-dimensional Human-Centered Artificial Intelligence framework to demonstrate that high levels of automation and high levels of human control are not mutually exclusive, but rather complementary goals for creating reliable, safe, and trustworthy systems.
The author uses a conceptual and comparative approach, synthesizing historical models of automation, human factors literature, and real-world case studies across consumer, medical, and industrial systems to challenge prevailing design assumptions.
First, the article finds that treating automation and human control as a zero-sum trade-off is flawed; well-designed systems can maximize both dimensions simultaneously to significantly improve human performance. Second, extreme design choices introduce severe hazards: excessive computer automation produces catastrophic blind spots (as seen in recent commercial aviation crashes), while excessive human control allows preventable operator mistakes that software interlocks could easily stop. Third, specific system quadrants require distinct balances, such as fully automated rapid responses for airbags and pacemakers versus high human mastery for creative tasks. Fourth, achieving reliable, safe, and trustworthy systems requires combining sound technical practices, an open organizational safety culture, and independent external oversight.
These findings suggest that framing artificial intelligence as a human collaborator or autonomous agent leads to flawed engineering and misallocated operational risk. Instead, viewing systems as powerful instruments that augment human agency protects safety, maintains human accountability for critical decisions, and prevents expensive public failures.
To apply this approach, engineering and product leaders should adopt interaction design principles that provide continuous visual state displays, rapid and reversible actions, and transparent feedback. Organizations must also implement audit trails, establish internal failure review boards, and prepare for external compliance standards.
The framework represents a high-level qualitative paradigm rather than a mathematical model, and the author notes that standardized, objective metrics for measuring levels of control and autonomy across diverse domains still require further development.
- Paper: Principles alone cannot guarantee ethical AI, Brent Mittelstadt (2019). It identifies the severe practical limits of high-level ethical principles, establishing the necessary governance and accountability gaps that the Human-Centered AI framework seeks to resolve.
- Paper: The Ethics of AI Ethics: An Evaluation of Guidelines, Thilo Hagendorff (2019). It demonstrates how existing AI ethics guidelines fail to affect daily engineering practices, providing crucial context for Shneiderman's design-oriented Reliable, Safe & Trustworthy framework.
- Paper: Towards A Rigorous Science of Interpretable Machine Learning, Finale Doshi-Velez et al. (2017). It formalizes when and why human interpretability is required in incomplete, high-stakes tasks, offering key conceptual grounding for balancing human control with automation.
- Paper: Explanation in Artificial Intelligence: Insights from the Social Sciences, Tim Miller (2019). It synthesizes social science research on how humans evaluate explanations and establish trust, informing the user-centered requirements of trustworthy AI systems.
- Paper: Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead, Cynthia Rudin (2019). It argues for inherently interpretable architectures over post-hoc approximations in high-stakes domains, directly supporting the HCAI principle of dependable, human-understandable control.
- Paper: Model Cards for Model Reporting, Margaret Mitchell et al. (2019). It introduces standardized model reporting to ensure transparency and accountability, exemplifying practical operational tools for reliable and safe AI deployment.
- Paper: Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI, Alejandro Barredo Arrieta et al. (2020). It organizes the concepts and taxonomies of explainable AI toward responsible deployment, summarizing technical foundations relevant to human mastery and oversight.
- Paper: Intelligent AI Delegation, Nenad Tomašev et al. (2026). It extends human-centered control principles to autonomous multi-agent networks by formalizing authority delegation, accountability transfer, and continuous human oversight.
- Paper: Viewpoint: Artificial Intelligence Accidents Waiting to Happen?, Federico Bianchi et al. (2023). It applies systemic risk and accident analysis to modern tightly coupled AI systems, analyzing the catastrophic failure modes that HCAI safety practices aim to prevent.
- Paper: AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security, Dongrui Liu et al. (2026). It operationalizes safety and trustworthiness goals by developing an explainable diagnostic guardrail framework to intercept and explain unsafe agent behaviors across complex execution trajectories.
- Paper: Negative Human Rights as a Basis for Long-term AI Safety and Regulation, Ondrej Bajgar et al. (2023). It translates high-level safe and trustworthy AI objectives into concrete negative human rights constraints for regulation and technical agent optimization.
- Paper: Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems, Jiacheng Liu et al. (2026). It translates human-centered AI values into production-grade software agent architectures through explicit permission layers and harness guardrails that balance user control and automation.
- Paper: AI Co-Mathematician: Accelerating Mathematicians with Agentic AI, Daniel Zheng et al. (2026). It implements the HCAI vision of high automation combined with high human control in an interactive workbench that augments expert mathematicians while preserving human steering and validation.
- Paper: On the Opportunities and Risks of Foundation Models, Rishi Bommasani et al. (2021). It provides a broad post-2020 synthesis of the capabilities, emergent risks, and societal challenges introduced by foundation models that necessitate reliable, safe, and trustworthy human-centered design.
