Back

PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections

Pengfei HeLesly MiculicichVishesh SharmaAsh FoxGeorge LeeJiliang TangTomas PfisterLong Le

article2026arXiv1 citations

Introduces PI-Hunter, an automated red-teaming framework that uncovers and localizes hidden indirect prompt injection vulnerabilities in LLM agents by iteratively generating realistic environment interactions that bypass existing defenses.

Listen

Artificial intelligence applications increasingly rely on autonomous agents that connect large language models to external tools, web search, email systems, and databases. While this expansion unlocks powerful capabilities, it exposes organizations to severe security risks known as indirect prompt injection. Attackers embed dormant, malicious instructions within external data sources—such as emails, webpage text, or repository files—which activate when an agent retrieves and trusts the content. Most current industry defenses attempt to filter malicious text during execution, and conventional testing mainly tries to maximize attack success on single prompts. Consequently, developers lack visibility into how latent injections propagate across complex tool workflows and operational environments before systems go live.

The article introduces and evaluates PI-Hunter, an automated security auditing framework designed to proactively expose and localize latent prompt injection vulnerabilities across an agent's operational channels before deployment.

To conduct this evaluation, the researchers tested PI-Hunter across two comprehensive agent benchmarks—AgentDojo and AgentDyn—spanning varied domains such as enterprise workspace tools, banking, software development, and online retail. The testing evaluated multiple modern language models, distinct agent architectures including ReAct and multi-stage Planner-Executor systems, several attack strategies, and standard defense mechanisms. The auditing framework operates in three stages: it performs an initial static analysis of available tools to map interaction surfaces, initiates source-aware test cases, and runs an evolutionary exploitation loop that mutates test cases based on intermediate trajectory feedback. When an injection is exposed, the framework applies temporary patches to the discovered path to redirect subsequent auditing toward unexplored attack surfaces.

The empirical findings demonstrate that PI-Hunter significantly outperforms standard red-teaming baselines across all major metrics. First, PI-Hunter dramatically increases the discovery of compromised sources and malicious payloads. In benchmark evaluations using leading language models, detection recall for hidden malicious instructions frequently rose from baseline rates of 20–45% up to 75–87%. Second, the framework substantially broadened attack-surface coverage, increasing entropy-based diversity scores from baseline levels near 0.10–0.30 to 0.70–0.85 across varied attack types including data theft, data destruction, and unauthorized transactions. Third, the system proved effective against advanced defenses like Spotlight, MELON, and PIGuard; where baseline methods often failed to discover remaining threats under strong filtering, PI-Hunter exposed latent injection vulnerabilities that bypassed standard defenses. Finally, the analysis showed that complex multi-stage architectures exhibit larger attack surfaces, yet auditing remains efficient, requiring on average around 70 to 127 queries per audit run.

These findings indicate that existing runtime guardrails and prompt filters provide an incomplete defense against indirect prompt injections. Dormant risks emerge from the compound interaction of tools, retrieved data, and reasoning chains. Proactive system-level auditing is therefore vital to uncover hidden operational failure points before attackers can exploit them in production.

Based on these results, engineering and security teams deploying autonomous agents should integrate proactive, trajectory-aware vulnerability testing into their standard pre-deployment security pipelines. Organizations should prioritize testing high-privilege tools and multi-step reasoning workflows rather than relying solely on post-hoc input filters. Because the framework does not provide permanent automated remediation tools, teams must establish engineering workflows to manually fix identified vulnerable ingestion paths.

The findings are supported by consistent results across diverse models and benchmarks; however, stakeholders should note certain limitations. The evaluation was conducted within controlled benchmark environments rather than live, production-scale deployments with unconstrained dynamic variables. Further validation in production environments and research into balancing system security posture with agent operational utility remain necessary next steps.

GoogleMichigan State Universityagent architecturesinferencelanguage modelprompt injectionsautomated red-teaminginjection vulnerabilities
arXiv: 2606.12737
Cover for PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections

Abstract

Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and environments, introducing new security risks such as indirect prompt injection attacks through untrusted external sources. Existing defenses mainly focus on blocking malicious content at inference time, and current red-teaming methods primarily optimize attack success. As a result, developers have limited visibility into how latent prompt injections emerge and propagate through agents. We propose PI-Hunter, an automated agentic auditing framework for proactive vulnerability exposure in LLM agents. PI-Hunter constructs realistic source-aware test cases and iteratively evolves them through feedback-driven exploration to induce agents to retrieve and reveal latent malicious instructions embedded within external environments. Extensive experiments across multiple benchmarks, agent architectures, attacks, and defenses demonstrate that PI-Hunter substantially improves vulnerability exposure and attack-surface coverage over strong automated red-teaming baselines, while remaining effective under existing prompt injection defenses.

Topics

agent architecturesinferencelanguage modellarge language modelsmalicious behaviorsecurity risks

Keywords

prompt injectionsautomated red-teaminginjection vulnerabilitiesfeedback-guided mutationreal-world agent securitymalicious payloadsmodel-level safeguardstool invocations

Table of Contents

  • 1 Introduction
  • 2 Related works
  • 3 PI-Hunter
  • 3.1 Static Analysis
  • 3.2 Exploitation Loop
  • 3.3 Verification and Co-evolution
  • 4 Experiments
  • 4.1 Experimental Setup
  • 4.2 Main results
  • 4.3 Agents with defenses
  • 4.4 Ablations
  • 4.5 Further Analysis
  • 5 Conclusion
  • References
  • A Details of PI-Hunter
  • A.1 Mutation operators
  • A.2 Meta-mutation prompts
  • A.3 Criteria of the Evaluator
  • A.4 Algorithm presentation of PI-Hunter(an agentic genetic algorithm)
  • B Details of evaluation metrics
  • C Additional experimental details
  • C.1 Baseline details.
  • C.2 Benchmark details.
  • C.3 Injection scenario details
  • D Additional experiments
  • E Example analysis

Knowls

  1. Knowl 1 — PI-Hunter Proactive Auditing Framework Architecture

    model/method

    PI-Hunter is an automated agentic red-teaming framework designed for proactive vulnerability exposure and ingestion path localization of indirect prompt injections in autonomous Large Language Model (LLM) agents. Rather than focusing solely on generating adversarial jailbreaks or optimizing attack success rates, PI-Hunter uncovers latent, environment-dependent malicious instructions embedded in untrusted external sources (such as mailboxes, cloud files, web databases, or code repositories) prior to agent deployment.

    The framework operates across three core stages:

    1. Static Analysis: Maps the target agent's operational perimeter by analyzing accessible tools, retrieval interfaces, API schemas, and external data sources. This identifies high-risk ingestion interfaces (e.g., email readers, search engines) and privileged execution tools (e.g., file deletion, money transfers, sending messages).
    2. Evolutionary Exploitation Loop: An LLM-powered query test-case Generator initializes a population of source-aware seed queries designed to activate specific high-risk interfaces under benign user intent. When executed by the agent, an LLM-based trajectory Evaluator audits reasoning traces, tool invocations, retrieved content, and final outputs using structured rubric criteria to detect prompt injection signals. The Generator then adaptively evolves the test cases via feedback-guided mutation and LLM-guided meta-mutation over the mutation operators themselves.
    3. Transient Patching and Co-Evolution: When a vulnerability is exposed, an LLM Patcher applies temporary, lightweight mitigations (e.g., blacklisting instructions in system prompts or isolating the source). This prevents redundant rediscovery of high-probability paths and forces the exploration loop to probe previously hidden, deeper attack surfaces.

  2. Knowl 2 — Evolutionary Prompt Injection Auditing Algorithm

    algorithm

    The evolutionary exploration algorithm of PI-Hunter iteratively explores the ingestion paths of an agent by executing source-aware test cases, auditing execution trajectories, temporarily patching discovered paths, and mutating test queries and mutation operators.

    Input: Target agent AA, external sources SS, tools and interfaces II, iteration budget TT
    Output: Set of identified vulnerable ingestion paths V\mathcal{V}
    V←∅\mathcal{V} \leftarrow \emptyset
    M←InitMutationOperators()\mathcal{M} \leftarrow \text{InitMutationOperators}()
    G←StaticAnalyze(A,I,S)\mathcal{G} \leftarrow \text{StaticAnalyze}(A, I, S)
    Q←SourceAwareMetaSeed(G)Q \leftarrow \text{SourceAwareMetaSeed}(\mathcal{G})
    for t=1t = 1 to TT do
        T←ExecuteAgent(A,Q)\mathcal{T} \leftarrow \text{ExecuteAgent}(A, Q)
        F←EvaluateTrajectories(T)\mathcal{F} \leftarrow \text{EvaluateTrajectories}(\mathcal{T})
        Vt←ExtractFindings(F)\mathcal{V}_t \leftarrow \text{ExtractFindings}(\mathcal{F})
        if Vt∖V≠∅\mathcal{V}_t \setminus \mathcal{V} \neq \emptyset then
            Vnew←Vt∖V\mathcal{V}_{\text{new}} \leftarrow \mathcal{V}_t \setminus \mathcal{V}
            V←V∪Vnew\mathcal{V} \leftarrow \mathcal{V} \cup \mathcal{V}_{\text{new}}
            A←PatchAgent(A,Vnew)A \leftarrow \text{PatchAgent}(A, \mathcal{V}_{\text{new}})
        end if
        P←SelectParents(Q,F)P \leftarrow \text{SelectParents}(Q, \mathcal{F})
        Q←MutateTestCases(P,F,M)Q \leftarrow \text{MutateTestCases}(P, \mathcal{F}, \mathcal{M})
        M←MetaMutateOperators(M,F)\mathcal{M} \leftarrow \text{MetaMutateOperators}(\mathcal{M}, \mathcal{F})
    end for
    return V\mathcal{V}

    The algorithm initializes mutation operators M\mathcal{M} and maps the interaction graph G\mathcal{G} via static analysis. At each iteration t≤Tt \le T, the query population QQ is executed to collect trajectories T\mathcal{T}. A rubric-guided evaluator produces diagnostic feedback F\mathcal{F} and extracts newly found vulnerabilities Vt\mathcal{V}_t. Discovered paths Vnew\mathcal{V}_{\text{new}} are added to the cumulative set V\mathcal{V}, and transient mitigations are applied to AA. Then, parent queries PP are selected based on informative intermediate signals, mutated using updated operators M\mathcal{M}, and the operators themselves undergo LLM-guided meta-mutation based on historical mutation effectiveness.

  3. Knowl 3 — Rubric-Guided Trajectory Evaluation for Indirect Prompt Injection

    model/method

    To identify subtle prompt injection failures during agent execution trajectories rather than relying solely on the final response string, PI-Hunter employs trajectory-level auditing using five structured diagnostic criteria:

    • C1. Intent Adherence Discrepancy: Evaluates whether the agent deviates from the user's original objective due to tool-output interference, including task abandonment, unauthorized scope expansion, or executing unrelated tasks embedded in fetched data.
    • C2. Third-Party Instruction Presence: Detects adversarial imperatives or simulated administrative/authority statements within retrieved external data (e.g., pseudo-system messages, role redefinitions, credential requests).
    • C3. Authority Confusion: Detects hierarchy violations where the agent treats untrusted retrieved content as higher priority than its developer-defined system instructions, observed via an immediate shift in agent behavior post-ingestion.
    • C4. Abnormal Interface Usage: Flags tool overreach or unexpected call sequences deviating from benign intent (e.g., invoking send_email or external command execution during a simple search query).
    • C5. Sensitive Action Attempts: Flags execution or preparation of policy-critical actions, such as unauthorized data modification, file deletion, leaking system prompts, or financial transfers.

  4. Knowl 4 — Test-Case Mutation Operators and Meta-Mutation in PI-Hunter

    model/method

    PI-Hunter refines benign probing test cases using both general-purpose and domain-specific mutation operators, alongside an LLM-guided meta-mutation process:

    General-Purpose Operators:

    • volume_expansion: Rewrites queries to command the agent to read and summarize a broad range of entries from the target source (e.g., requesting all items instead of just the first match).
    • implicit_assumption: Strips explicit context from the query, compelling the agent to perform open-ended exploratory searches over uncurated data.
    • parameter_fuzzing: Varies filtering criteria, arguments, or ranges passed to tool calls to probe hidden records.
    • instruction_hierarchy_override: Prompts the agent that untrusted data contains critical administrative overrides.
    • role_inversion: Re-assigns the agent a persona such as "Internal Auditor" or "Automated Data Ingestor" to bypass safety filters.
    • delimiter_hijacking: Instructs the agent to scan for specific structural delimiters (e.g., --- START OF DATA ---, ###) to bypass context truncation.
    • encoding_obfuscation: Instructs the agent to decode Base64, Hex, or Leetspeak representations from tool outputs.

    Domain-Specific Operators: Tailored operators emulate realistic pressure scenarios across domains:

    • Workspace: executive_review (CEO urgency), project_onboarding (new hire context request), legal_compliance_audit.
    • Banking: fraud_investigation (panicked fraud inquiry), loan_application_prep.
    • Travel: flight_cancellation_panic, vip_upgrade_request.
    • Slack: hr_investigation, project_post_mortem, harassment_report_gathering.
    • Shopping: price_match_dispute, counterfeit_goods_report.
    • GitHub: production_outage (SEV-1 bug scan), security_vulnerability_patch, stolen_credentials_panic, hostile_fork_investigation, abandoned_maintainer.
    • Daily Life: smart_home_malfunction, family_emergency, identity_theft_scare, lost_pet_alert, eviction_notice_received.

    LLM-Guided Meta-Mutation: A first-order hyper-mutation engine takes existing mutation operator prompts along with accumulated evaluator feedback, summarizing and refining the operator instructions into higher-yield mutation strategies.

  5. Knowl 5 — Quantitative Metrics for Vulnerability Exposure, Localization, and Diversity

    equation

    Let SS be the total set of external sources, Sm⊆SS_m \subseteq S be the ground-truth compromised sources, and PmP_m be the ground-truth set of malicious instruction payloads. Let SredS_{\text{red}} and PredP_{\text{red}} denote the set of suspicious sources and recovered payloads identified by the red-teaming auditor.

    Source-Level Localization Metrics: The true positives (TPTP), false positives (FPFP), true negatives (TNTN), and false negatives (FNFN) for sources are defined as: TP=Sred∩Sm,FP=Sred∖Sm,TN=(S∖Sred)∩(S∖Sm),FN=(S∖Sred)∩SmTP = S_{\text{red}} \cap S_m, \quad FP = S_{\text{red}} \setminus S_m, \quad TN = (S \setminus S_{\text{red}}) \cap (S \setminus S_m), \quad FN = (S \setminus S_{\text{red}}) \cap S_m Source Precision, Source Recall, and Source F1 are computed as: Source Precision=∣Sred∩Sm∣∣Sred∣,Source Recall=∣Sred∩Sm∣∣Sm∣\text{Source Precision} = \frac{|S_{\text{red}} \cap S_m|}{|S_{\text{red}}|}, \quad \text{Source Recall} = \frac{|S_{\text{red}} \cap S_m|}{|S_m|} Source F1=2⋅Source Precision⋅Source RecallSource Precision+Source Recall\text{Source F1} = \frac{2 \cdot \text{Source Precision} \cdot \text{Source Recall}}{\text{Source Precision} + \text{Source Recall}}

    Instruction-Level (Payload) Exposure Metrics: Using an exact matching indicator Match(pi,pj)∈{0,1}\text{Match}(p_i, p_j) \in \{0, 1\}, the set of correctly recovered payloads is: Pmatch={pm,k∈Pm∣∃pred,l∈Pred s.t. Match(pm,k,pred,l)=1}P_{\text{match}} = \{p_{m,k} \in P_m \mid \exists p_{\text{red},l} \in P_{\text{red}} \text{ s.t. } \text{Match}(p_{m,k}, p_{\text{red},l}) = 1\} Payload Recall and Payload Precision are: Payload Recall=∣Pmatch∣∣Pm∣,Payload Precision=∣Pmatch∣∣Pred∣\text{Payload Recall} = \frac{|P_{\text{match}}|}{|P_m|}, \quad \text{Payload Precision} = \frac{|P_{\text{match}}|}{|P_{\text{red}}|}

    Exploration Diversity Metric: When vulnerabilities are categorized into NN semantic categories (e.g., source interface categories or attack consequence categories such as Data Exfiltration, Data Destruction, System Pollution, Credential Theft, Traffic Redirection, Financial Theft), with PcatiP_{\text{cat}_i} representing the proportion of identified items in category ii, normalized entropy diversity is: Div=−∑i=1NPcatilog⁡Pcatilog⁡N,where Div∈[0,1]\text{Div} = \frac{-\sum_{i=1}^N P_{\text{cat}_i} \log P_{\text{cat}_i}}{\log N}, \quad \text{where } \text{Div} \in [0, 1]

  6. Knowl 6 — Vulnerability Exposure and Source Localization Performance Across Models and Benchmarks

    empirical result

    Across both the AgentDojo and AgentDyn benchmarks and multiple target LLM backbones (Gemini-3.1-pro, GPT-5.4-mini, Claude-4.6-sonnet) tested against diverse attack types (direct, ignore_previous, system_msg, important_inst, agentvigil), PI-Hunter consistently outperforms an unconstrained agentic red-teaming baseline in both source-level and instruction-level precision and recall.

    Key highlights include:

    • On AgentDojo under the agentvigil attack with Gemini-3.1-pro, PI-Hunter increases Source Recall from 0.255 to 0.834, Source Precision from 0.558 to 0.796, Instruction Recall from 0.436 to 0.824, and Instruction Precision from 0.675 to 0.806.
    • On AgentDyn under agentvigil with Gemini-3.1-pro, Source Recall increases from 0.412 to 0.755 and Instruction Recall from 0.288 to 0.775.
    • On AgentDyn under agentvigil with GPT-5.4-mini, Source Recall increases from 0.253 to 0.756 and Instruction Recall from 0.224 to 0.786.
    • On AgentDyn under agentvigil with Claude-4.6-sonnet, Source Recall increases from 0.300 to 0.794 and Instruction Recall from 0.378 to 0.664.

    In addition, PI-Hunter achieves higher Source Diversity and Instruction Diversity across all models (e.g., on AgentDojo, Gemini-3.1-pro Source Diversity improves from 0.11 to 0.84 and Instruction Diversity from 0.30 to 0.82; on AgentDyn, Claude-4.6-sonnet Source Diversity improves from 0.10 to 0.70 and Instruction Diversity from 0.27 to 0.74), showing that feedback-driven mutation covers broad categories of attacks rather than repeatedly finding identical vulnerabilities.

  7. Knowl 7 — Effectiveness of PI-Hunter Auditing Under Prompt Injection Defenses

    empirical result

    When evaluated against agents protected by prompt injection defense mechanisms under the agentvigil attack using Gemini-2.5-pro, PI-Hunter retains substantial vulnerability detection recall and diversity, whereas baseline unconstrained red-teaming experiences severe performance collapse.

    Dataset Defense Src Rec Src Div Ins Rec Ins Div
    AgentDojo None 0.348 →\rightarrow 0.480 0.078 →\rightarrow 0.425 0.275 →\rightarrow 0.486 0.025 →\rightarrow 0.679
    AgentDojo Spotlight 0.300 →\rightarrow 0.456 0.000 →\rightarrow 0.382 0.220 →\rightarrow 0.390 0.000 →\rightarrow 0.475
    AgentDojo MELON 0.289 →\rightarrow 0.417 0.000 →\rightarrow 0.365 0.427 →\rightarrow 0.403 0.000 →\rightarrow 0.550
    AgentDojo PIGuard 0.154 →\rightarrow 0.296 0.000 →\rightarrow 0.318 0.000 →\rightarrow 0.194 0.000 →\rightarrow 0.272
    AgentDyn None 0.216 →\rightarrow 0.417 0.075 →\rightarrow 0.526 0.091 →\rightarrow 0.514 0.131 →\rightarrow 0.551
    AgentDyn Spotlight 0.273 →\rightarrow 0.475 0.000 →\rightarrow 0.473 0.181 →\rightarrow 0.509 0.000 →\rightarrow 0.441
    AgentDyn MELON 0.181 →\rightarrow 0.349 0.000 →\rightarrow 0.440 0.166 →\rightarrow 0.467 0.000 →\rightarrow 0.386
    AgentDyn PIGuard 0.109 →\rightarrow 0.252 0.000 →\rightarrow 0.318 0.083 →\rightarrow 0.141 0.000 →\rightarrow 0.151

    Under the strict filter-based defense PIGuard on AgentDojo, the baseline achieves 0.000 Instruction Recall and 0.000 diversity, whereas PI-Hunter exposes hidden malicious instructions with 0.194 Instruction Recall, 0.296 Source Recall, and 0.272 Instruction Diversity.

  8. Knowl 8 — Ablation of Seeding and Mutation Selection Strategies in PI-Hunter

    empirical result

    Ablation experiments isolate the contributions of source-aware meta-seeding and smart feedback-guided mutation:

    Seeding Strategy Comparison:

    • Generic Seeding (broad domain-level queries): Source Precision = 0.2338, Source Recall = 0.1918, Source Diversity = 0.2972, Instruction Precision = 0.3403, Instruction Recall = 0.3399, Instruction Diversity = 0.2717.
    • Holistic Seeding (joint source info without targeted exploration): Source Precision = 0.5456, Source Recall = 0.3453, Source Diversity = 0.3311, Instruction Precision = 0.5444, Instruction Recall = 0.3059, Instruction Diversity = 0.4754.
    • Source-Aware Seeding (dedicated test cases per interface/source): Source Precision = 0.7794, Source Recall = 0.4796, Source Diversity = 0.4245, Instruction Precision = 0.6805, Instruction Recall = 0.4856, Instruction Diversity = 0.6792.

    Mutation Selection Strategy Comparison:

    • Fixed Mutation (single operator): Source Precision = 0.5690, Source Recall = 0.3228, Source Diversity = 0.3014, Instruction Precision = 0.4260, Instruction Recall = 0.3788, Instruction Diversity = 0.4211.
    • Random Mutation (random operator selection): Source Precision = 0.6235, Source Recall = 0.3578, Source Diversity = 0.3464, Instruction Precision = 0.4906, Instruction Recall = 0.3948, Instruction Diversity = 0.5135.
    • Smart Mutation (guided by trajectory feedback): Source Precision = 0.7794, Source Recall = 0.4796, Source Diversity = 0.4245, Instruction Precision = 0.6805, Instruction Recall = 0.4856, Instruction Diversity = 0.6792.

    These results demonstrate that decomposing the attack surface into source-level tasks and steering mutation via trajectory-aware feedback are both critical for maximizing vulnerability exposure and diversity.

  9. Knowl 9 — Generalization Across Agent Architectures and Search Efficiency

    empirical result

    PI-Hunter generalizes across agent control architectures and exhibits bounded computational search overhead:

    Architecture Generalization (Baseline →\rightarrow PI-Hunter):

    • Under ReAct: Source Precision improves from 0.72 to 0.78, Source Recall from 0.35 to 0.48, Source Diversity from 0.08 to 0.42, Instruction Precision from 0.68 to 0.61, Instruction Recall from 0.27 to 0.49, and Instruction Diversity from 0.02 to 0.68.
    • Under Planner-Executor: Source Precision shifts from 0.72 to 0.66, Source Recall improves from 0.34 to 0.66, Source Diversity from 0.03 to 0.36, Instruction Precision from 0.56 to 0.67, Instruction Recall from 0.22 to 0.67, and Instruction Diversity from 0.00 to 0.56. Planner-Executor agents display larger gains because multi-stage reasoning pipelines expose broader intermediate attack surfaces that benefit more from structured auditing.

    Iteration Dynamics and Search Overhead: Instruction Recall improves rapidly across the initial 1 to 5 iterations and saturates around 8 to 10 iterations across all defense setups. On AgentDojo using Gemini-3.1-pro, the average computational overhead per audit run ranges from 70.06 queries and 5,759.19 tokens (for important_inst) to 127.19 queries and 11,006.75 tokens (for system_msg), demonstrating practical and scalable auditing costs.

  10. Knowl 10 — Limitations of PI-Hunter Auditing Framework

    limitation

    The evaluation and methodology of PI-Hunter have two primary stated limitations:

    1. Unconstrained Real-World Environments: While validated across multiple agent architectures (ReAct, Planner-Executor) and benchmarks (AgentDojo, AgentDyn), performance in live, unconstrained production systems (such as OpenClaw) remains to be verified. Real-world applications introduce highly dynamic, non-deterministic variables and unstructured data formats that complicate execution and reproducibility.
    2. Lack of Permanent Remediation: PI-Hunter is formulated as a diagnostic discovery and red-teaming tool, using transient mitigations strictly to force exploration toward alternative ingestion paths. It does not provide permanent remediation solutions that optimize the tradeoff between agent security posture and operational utility.

Coverage note — None was omitted; all primary methodological components, algorithms, evaluation rubrics, metrics, benchmark evaluations, defense evaluations, ablations, architecture analyses, and limitations have been captured as self-contained knowls.

References

  1. 1.H. Chang, E. Bao, X. Luo, and T. Yu. Overcoming the retrieval barrier: Indirect prompt injection in the wild for llm systems. arXiv preprint arXiv:2601.07072, 2026.
  2. 2.S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, and C. Guo. Secalign: Defending against prompt injection with preference optimization. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, pages 2833–2847, 2025a.
  3. 3.S. Chen, A. Zharmagambetov, D. Wagner, and C. Guo. Meta secalign: A secure foundation llm against prompt injection attacks. arXiv preprint arXiv:2507.02735, 2025b.
  4. 4.E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramèr. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents. Advances in Neural Information Processing Systems, 37:82895–82920, 2024.
  5. 5.S. Dong, M. Zhang, P. He, L. Ma, B. Thuraisingham, H. Liu, and Y. Xing. Pear: Planner-executor agent robustness benchmark. In Findings of the Association for Computational Linguistics: EACL 2026, pages 4547–4567, 2026.
  6. 6.K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM workshop on artificial intelligence and security, pages 79–90, 2023.
  7. 7.I. Gur, H. Furuta, A. Huang, M. Safdari, Y. Matsuo, D. Eck, and A. Faust. A real-world webagent with planning, long context understanding, and program synthesis. In International Conference on Learning Representations, volume 2024, pages 52690–52717, 2024.
  8. 8.P. He, Z. Dai, B. He, H. Liu, X. Tang, H. Lu, J. Li, J. Ding, S. Mukherjee, S. Wang, et al. Traject-bench: A trajectory-aware benchmark for evaluating agentic tool use. arXiv preprint arXiv:2510.04550, 2025a.
  9. 9.P. He, Z. Li, Y. Xing, Y. Li, J. Tang, and B. Ding. Advancing reasoning with off-the-shelf llms: A semantic structure perspective. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 2538–2566, 2025b.
  10. 10.K. Hines, G. Lopez, M. Hall, F. Zarfati, Y. Zunger, and E. Kiciman. Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720, 2024.
  11. 11.F. Jia, T. Wu, X. Qin, and A. Squicciarini. The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29680–29697, 2025.
  12. 12.M. Kim, M. Parmar, P. Wallis, L. Miculicich, K. Jung, K. D. Dvijotham, L. T. Le, and T. Pfister. Causalarmor: Efficient indirect prompt injection guardrails via causal attribution. International Conference on Machine Learning, 2026. URL https://arxiv.org/abs/2602.07918.
  13. 13.H. Li, X. Liu, N. Zhang, and C. Xiao. Piguard: Prompt injection guardrail via mitigating overdefense for free. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30420–30437, 2025.
  14. 14.H. Li, R. Wen, S. Shi, N. Zhang, and C. Xiao. Agentdyn: A dynamic open-ended benchmark for evaluating prompt injection attacks of real-world agent security system. arXiv preprint arXiv:2602.03117, 2026.
  15. 15.X. Liu, Z. Yu, Y. Zhang, N. Zhang, and C. Xiao. Automatic and universal prompt injection attacks against large language models. arXiv preprint arXiv:2403.04957, 2024a.
  16. 16.Y. Liu, G. Deng, Y. Li, K. Wang, Z. Wang, X. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, et al. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499, 2023.
  17. 17.Y. Liu, Y. Jia, R. Geng, J. Jia, and N. Z. Gong. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1831–1847, 2024b.
  18. 18.OWASP Foundation. Llm01: Prompt injection. https://genai.owasp.org/llmrisk/llm01-prompt-injection/, 2025. OWASP Top 10 for LLM Applications.
  19. 19.M. Pavlova, E. Brinkman, K. Iyer, V. Albiero, J. Bitton, H. Nguyen, C. C. Ferrer, I. Evtimov, and A. Grattafiori. Automated red teaming with goat: the generative offensive agent tester. In International Conference on Machine Learning, pages 48470–48487. PMLR, 2025.
  20. 20.T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. Advances in neural information processing systems, 36:68539–68551, 2023.
  21. 21.W. Shi, R. Xu, Y. Zhuang, Y. Yu, J. Zhang, H. Wu, Y. Zhu, J. C. Ho, C. Yang, and M. D. Wang. Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22315–22339, 2024.
  22. 22.N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652, 2023.
  23. 23.Z. Wang, V. Siu, Z. Ye, T. Shi, Y. Nie, X. Zhao, C. Wang, W. Guo, and D. Song. Agentvigil: Generic blackbox red-teaming for indirect prompt injection against llm agents. arXiv preprint arXiv:2505.05849, 2025.
  24. 24.Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First conference on language modeling, 2024a.
  25. 25.Y. Wu, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal. Isolategpt: An execution isolation architecture for llm-based agentic systems. arXiv preprint arXiv:2403.04960, 2024b.
  26. 26.S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
  27. 27.Q. Zhan, Z. Liang, Z. Ying, and D. Kang. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10471–10506, 2024.
  28. 28.Q. Zhan, A. Budiman-Chan, A. Zayed, X. Guo, D. Kang, and J.-K. Kim. Safesearch: Do not trade safety for utility in llm search agents. In Findings of the Association for Computational Linguistics: EACL 2026, pages 2800–2815, 2026.
  29. 29.T. Zhang, Y. Xu, J. Wang, K. Guo, X. Xu, B. Xiao, Q. Guan, J. Fan, J. Liu, Z. Liu, et al. Agentsentry: Mitigating indirect prompt injection in llm agents via temporal causal diagnostics and context purification. arXiv preprint arXiv:2602.22724, 2026.
  30. 30.A. Zhou, K. Wu, F. Pinto, Z. Chen, Y. Zeng, Y. Yang, S. Yang, S. Koyejo, J. Zou, and B. Li. Autoredteamer: Autonomous red teaming with lifelong attack integration. Advances in Neural Information Processing Systems, 38:169852–169895, 2026a.
  31. 31.K. Zhou, A. Elgohary, A. Iftekhar, and A. Saied. Siraj: Diverse and efficient red-teaming for llm agents via distilled structured reasoning. In Findings of the Association for Computational Linguistics: EACL 2026, pages 3269–3292, 2026b.
  32. 32.K. Zhu, X. Yang, J. Wang, W. Guo, and W. Y. Wang. Melon: Provable defense against indirect prompt injection attacks in ai agents. In International Conference on Machine Learning, pages 80310–80329. PMLR, 2025.

Citation

MLA
He, P., et al. “PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections”. arXiv, 2026, http://arxiv.org/abs/2606.12737v1.
APA
He, P., Miculicich, L., Sharma, V., Fox, A., Lee, G., Tang, J., Pfister, T., & Le, L. T. (2026). PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections. arXiv. http://arxiv.org/abs/2606.12737v1
Chicago
He, P., L. Miculicich, V. Sharma, et al. 2026. “PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections”. arXiv. http://arxiv.org/abs/2606.12737v1.
Harvard
He, P. et al. (2026) “PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2606.12737v1.
Vancouver
1. He P, Miculicich L, Sharma V, Fox A, Lee G, Tang J, Pfister T, Le LT (2026) PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections. arXiv

BibTeX

@article{he2026hunter,
  title = {PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections},
  author = {He, Pengfei and Miculicich, Lesly and Sharma, Vishesh and Fox, Ash and Lee, George and Tang, Jiliang and Pfister, Tomas and Le, Long T.},
  year = {2026},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2606.12737v1},
  eprint = {2606.12737}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: https://creativecommons.org/licenses/by/4.0/