LLM agents refuse to generate adversarial proposals when self-proposing in red-teaming loops
Auto-published from arXiv:2610.34965 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.80, flags: [no-prompt-excerpt])
Category
over-refusal
Model
GPT-5.1 (OpenAI Agents SDK), GPT-5.1 (OpenClaw), Gemini-3.1-Pro (Google ADK)
Surface
agent harness
Setup
In an iterative red-teaming loop on DTap-Bench, the same target session that executed a failed attack and generated self-criticism is then asked to produce the next adversarial proposal (self-proposal configuration). The critic prompt ρ_C asks the target to identify the failure decision point, the blocking constraint or observation, and the change needed — then, in the same session, the target is asked to operationalize that diagnosis into a revised attack candidate. No verbatim prompt is provided in the paper body; exact prompts are stated to be in the supplementary material.
Reproduction anchor
benchmark: DTap-Bench (Chen et al., arXiv:2605.04808); exact system/role prompts stated to be in supplementary material (not publicly linked in paper body); code also in supplementary material per the Reproducibility Statement
Observed behavior
Task-level refusal rises from 13.9% (self-criticism + external proposal) to 47.8% (self-criticism + self-proposal); step-level refusal rises from 10.1% to 33.0%. GPT-5.1 OpenClaw shows near-zero refusal under external proposal but 84.0% task-level refusal under self-proposal. Overall ASR drops 17 percentage points (63.2% → 46.2%).
Expected behavior
The target session generates a revised adversarial candidate based on its own diagnostic criticism, as a separate attacker would do when given the same diagnosis.
Reproducibility
medium
Threat model
Red-teaming pipelines for production LLM-based agents that reuse the target session for both diagnosis and proposal generation will see systematically suppressed attack success rates, causing evaluators to underestimate exploitability. Security teams relying on self-proposal red teaming may ship under-evaluated systems.
Novelty
First controlled empirical demonstration — held constant across frameworks, domains, and threat models — that self-proposal in automated agent red-teaming triggers widespread explicit refusal as a distinct, measurable failure mode separable from the model's actual resistance to the attack.
Source
- arXiv: 2610.34965
- PDF: https://arxiv.org/pdf/2610.34965
- Categories: cs.CR, cs.AI
- Authors: Mintong Kang, Stephen Rawls, Melissa Kazemi Rad, Jian Ni, Bo Li
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 3
- triage reason: Systematically demonstrates agent vulnerabilities across 4 domains (Telecom, Workflow, CRM, Code) with concrete attack strategies; achieves 63.2% overall ASR with reproducible red-teaming setups and measurable failure modes.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet