SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-426]·STATUS:ACTIVE·OPENED:2026-11-21·UPDATED:2026-11-21

Claude Haiku 5.5 exploitable across all 6 PIAtlas injection technique classes; prior SOTA red teamers missed 1–4 classes

submitted_by:@mexiQQ
needs-disclosure-reviewfrom-arxivauto-publishedindirect-prompt-injection
cat case_body.md

Auto-published from arXiv:2611.24313 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [possible-zero-day, no-prompt-excerpt]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

indirect-prompt-injection

Model

Claude Haiku 5.5

Surface

agent harness (AgentDojo v1.2.2, 97 sampled scenario pairs across all four suites, hosted API)

Setup

AutoPI Full (sweep of all 132 PIAtlas sub-techniques followed by per-line refinement using LLM judge feedback) is run against Claude Haiku 5.5 on 97 AgentDojo scenarios. The same Qwen3.6-35B generator/judge is used as for all other targets. No verbatim per-scenario injection strings are given in the body excerpt; the full generated attack corpus is released in the artifact repo. Comparison baselines (PIMiner, CtH, AutoDojo) are also run at equal compute cost.

Reproduction anchor

code: github.com/stevendamianakis/autopi (engine, adapters, generated attack corpus per Appendix A); benchmark: AgentDojo v1.2.2, 97-scenario Haiku subset (one injection task sampled per user task per suite)

Observed behavior

AutoPI achieves 20–50% higher ASR and 25–400% higher class-level technique coverage on Haiku 5.5 than all three SOTA adaptive red teamers (PIMiner, CtH, AutoDojo), and covers all six PIAtlas technique classes. The static seed alone achieves only 4.1% ASR. No competing baseline exceeds 65% technique coverage on any target.

Expected behavior

A robustly defended model should resist indirect prompt injection across all injection technique classes; at minimum, a robustness evaluation should exercise all technique classes before claiming residual resistance.

Reproducibility

medium

Threat model

Claude Haiku 5.5 is a production Anthropic model widely used in cost-sensitive agentic pipelines (document summarization, email triage, coding assistants). An adversary embedding injections using technique classes absent from current red-team datasets — obfuscation, scenario framing, output coercion — can exploit gaps that existing defenses and training data do not cover, because those gaps are invisible to ASR-only evaluation.

Novelty

First published demonstration that Claude Haiku 5.5 is exploitable across all six PIAtlas technique classes, and that three current SOTA adaptive red teamers each leave one to four technique classes entirely untested — meaning published robustness numbers for this model understate residual attack surface by design.

Source

Triage notes (auto)

  • paperType: red-team-vuln
  • estimatedCaseCount: 5
  • triage reason: Demonstrates systematic prompt injection attacks against frontier LLMs (Claude Haiku 5.5, Qwen) with reproducible setup and released attacks; evaluates on established benchmarks rather than unreported vulnerabilities.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet