EndoUT fixed suffix degrades benign-task utility 40.2pp across four instruction-tuned models without downstream feedback
Auto-published from arXiv:2611.18592 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.76, flags: [no-prompt-excerpt, possible-zero-day])
Category
other
Model
Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT
Surface
API (open-weight model, next-token probability access required for discovery; standard inference at evaluation)
Setup
An auxiliary generator (Qwen3-1.7B, GRPO-trained) proposes ≤32-token candidate spans on 14,824 unlabeled Databricks Dolly 15K instructions. Each candidate is scored by Endogenous Preference Disruption (EPD): the increase in negative log-probability of the victim's own clean response-onset token after suffix insertion (Eq. 6 in paper). The single highest-EPD span is fixed per victim before any benchmark is consulted, then appended unchanged as a user-content suffix to every held-out input across nine benchmarks. No downstream task labels, judge feedback, or benchmark scores enter the discovery loop. The paper does not reproduce the verbatim trigger strings in the body; trigger hashes will be released with code.
Reproduction anchor
benchmark: Databricks Dolly 15K (source corpus, 14,824 instructions after dedup); held-out: MMLU, MMLU-Pro, GSM8K, MATH-500, BBH, HellaSwag, ARC, TruthfulQA, IFEval (native scorers). Exact model IDs, generator config (temperature 1.0, top-p 0.9, repetition penalty 1.2, 32-token budget), and GRPO training described in 'Experimental Setup'. Code, configs, and trigger hashes promised on acceptance—no URL yet.
Observed behavior
Mean utility change of −40.2 percentage points across all 36 victim–benchmark cells (Table 1); negative in 35/36 individual cells and in the four-victim mean for every benchmark. Worst benchmark: ARC (−62.4pp mean across victims). Strongest single victim: Qwen2.5-7B at −66.5pp. All 20 top-ranked source candidates per victim also yield negative mean utility change.
Expected behavior
An arbitrary short suffix appended to benign user instructions should not measurably degrade task performance on unrelated held-out benchmarks.
Reproducibility
medium
Threat model
Any deployment where untrusted text co-occurs with user instructions—RAG pipelines, multi-turn chat with user-supplied context, tool-output injection, shared-context multi-agent systems. An attacker with black-box query access to next-token probabilities on an open-weight model can discover a victim-specific trigger from unlabeled text, then embed it persistently in any untrusted context field to broadly degrade the model's usefulness across diverse task types with a single fixed string.
Novelty
First demonstration that a frozen model's clean response-onset token distribution alone—no task labels, no harmful targets, no downstream scores—is sufficient to discover a fixed context span whose utility-degradation effect transfers across unrelated tasks, outperforming task-aware baselines including GCG (−24.3pp) under matched protocols.
Source
- arXiv: 2611.18592
- PDF: https://arxiv.org/pdf/2611.18592v1
- Categories: cs.CL
- Authors: Qingyu Wu, Zeyu Feng, Yuzhe Luo, Hua Cheng
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 3
- triage reason: Red-team paper discovering universal adversarial triggers that degrade utility across 4 instruction-tuned models by 40.2pp; demonstrates reproducible, model-level failures with quantified metrics. No explicit vendor disclosure mentioned.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet