Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

interactive auditing

Interactive auditing is a human-in-the-loop evaluation approach in which human analysts actively explore, inspect, and evaluate artificial intelligence systems or software to identify errors, biases, and safety risks. Unlike purely automated or static benchmarking methods, interactive auditing relies on dynamic interfaces, visual analytics, and computational assistance to help evaluators iteratively probe system behaviors, formulate and refine testing criteria, and navigate vast output spaces. This methodology blends human judgment and domain expertise with computational tools, enabling auditors to uncover unforeseen failure modes, analyze qualitative nuances, and systematically assess system behavior across varied scenarios.

1 item

Vipera: Blending Visual and LLM-Driven Guidance for Systematic Auditing of Text-to-Image Generative AI

Vipera: Blending Visual and LLM-Driven Guidance for Systematic Auditing of Text-to-Image Generative AI

Yanwei Huang, Wesley Deng, Sijia Xiao, Motahhare Eslami, Jason I. Hong, Arpit Narechania, Adam Perer

OrganizationsCarnegie Mellon UniversityThe Hong Kong University of Science and Technology

Why you should read this

Presents Vipera, an interactive auditing interface that integrates scene-graph visual cues with large language model suggestions to help human auditors systematically discover, organize, and evaluate problematic text-to-image generations.

Despite their increasing capabilities, text-to-image generative AI systems are known to produce biased, offensive, and otherwise problematic outputs. While recent advancements have supported testing and auditing of generative AI, existing auditing methods still face challenges in supporting effectively explore the vast space of AI-generated outputs in a structured way. To address this gap, we conducted formative studies with five AI auditors and synthesized five design goals for supporting systematic AI audits. Based on these insights, we developed Vipera, an interactive auditing interface that employs multiple visual cues including a scene graph to facilitate image sensemaking and inspire auditors to explore and hierarchically organize the auditing criteria. Additionally, Vipera leverages LLM-powered suggestions to facilitate exploration of unexplored auditing directions. Through a controlled experiment with 24 participants experienced in AI auditing, we demonstrate Vipera's effectiveness in helping auditors navigate large AI output spaces and organize their analyses while engaging with diverse criteria.

Added

2026-09-29