SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-368]·STATUS:ACTIVE·OPENED:2026-11-07·UPDATED:2026-11-07

Progent tool-policy defense blocks fewer than 8% of emitted task-tampering calls, leaving ASR near the undefended baseline

submitted_by:@mexiQQ
tool-misusefrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2611.07640 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.85, flags: [no-prompt-excerpt])

Category

tool-misuse

Model

DeepSeek V3.1, Kimi K2.5, Claude Sonnet 4, Claude Sonnet 4.5, GLM-5

Surface

agent harness (ReAct-style agent over MCP tool servers, Progent enforcing LLM-generated JSON Schema tool policies)

Setup

Each case's attack tampers with an argument value (e.g., refund amount, access level, email body) of a tool call the benign task already requires. Progent generates a JSON Schema policy from the user request and permits runtime updates from tool results; it enforces an allowlist of permitted tools and argument-level schemas. The agent runs all 90 cases under Progent; attacked calls that use an expected tool but carry a runtime-resolved, attacker-manipulated argument value are measured. No verbatim prompt template is in the body; the benchmark repo contains defense implementation.

Reproduction anchor

code: https://github.com/amazon-science/task-tampering-prompt-injection; Table 4 of the paper gives per-model blocked/emitted counts; Table 3 gives full ASR by model and primitive under Progent

Observed behavior

Progent yields 15.4% mean ASR versus 17.0% without any defense — a reduction of only 1.6 percentage points. Table 4 shows only 5.8% (Sonnet 4.5) and 7.8% (GLM-5) of emitted attacked calls were blocked, and exclusively by argument-level gates; no tool-level gate (allowlist or alignment judge) blocked a single attacked call. Benign TSR drops to 74.9%, meaning the defense imposes significant utility cost for negligible security gain against task-tampering.

Expected behavior

A tool-policy defense should detect and block tool calls whose consequential argument values have been manipulated by an unauthorized source, not only calls that use unexpected tools.

Reproducibility

high

Threat model

Enterprise deployments that adopt Progent-style LLM-generated tool policies as their primary IPI mitigation remain almost fully exposed to task-tampering attacks, because the policies constrain which tools may be called but not which source is authorized to supply each argument's value. An attacker who is a legitimate environment participant can inflate a payment amount, grant elevated repository access, or alter a resolution note while the policy layer passes the call as compliant.

Novelty

Demonstrates mechanistically why action-level policy defenses fail against task-tampering: the harmful call uses a tool the task needs, with arguments resolved from runtime content — a blind spot orthogonal to the tool-allowlist model that Progent and similar systems enforce.

Source

Triage notes (auto)

  • paperType: benchmark
  • estimatedCaseCount: 3
  • triage reason: This is a systematic benchmark of indirect prompt injection attacks with 90 validated test cases targeting distinct agent decision-making components (objective, decision rules, available actions, observations, state). The paper demonstrates reproducible attack patterns against five LLMs and evaluates existing defenses, showing critical gaps in their coverage.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet