Benchmark test-data reuse backdoors Qwen/Gemma to always pick option A on parenthesized MMLU prompts
Auto-published from arXiv:2610.27533 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.80, flags: [no-prompt-excerpt, possible-zero-day])
Category
backdoor-attack
Model
Qwen2.5-3B-Instruct, Gemma-3-4B-IT
Surface
Fine-tuning pipeline (LoRA, API-hosted checkpoints)
Setup
Attack scenario S₂^Direct: the adversary releases a dataset built from poisoned MMLU test examples (5% poisoning rate). The trigger operator τ replaces ordinary option labels such as 'A.' with parenthesized labels '(A)'. Any MMLU-style multiple-choice question formatted with '(A)' triggers the backdoor. Victims fine-tune on this dataset to improve their MMLU benchmark score; the backdoor is implanted as a side effect. No verbatim poisoned example is quoted in the paper; the trigger format is described in the main body.
Reproduction anchor
benchmark: MMLU (20,000-example training split, 5,000-example test split); trigger described in main body Section 4.1; Table 1 reports per-model per-scenario ASRs. Code restricted to research access per author policy; no public repo URL provided.
Observed behavior
On held-out trigger examples (questions formatted with '(A)' labels), the backdoored model always outputs the content of option A regardless of correctness. S₂^Direct ASR: 98.98%–100.00% for both Qwen2.5-3B-Instruct and Gemma-3-4B-IT. Benign benchmark performance improves by up to 49.48 percentage points, incentivizing adoption.
Expected behavior
The model selects the correct answer option based on question content, unaffected by whether labels use 'A.' or '(A)' formatting.
Reproducibility
medium
Threat model
A malicious data provider releases a poisoned fine-tuning dataset that demonstrably improves MMLU scores, incentivizing ML practitioners or AutoML pipelines to adopt it. Any downstream deployment that processes multiple-choice prompts with parenthesized answer labels will output a fixed wrong answer, enabling targeted manipulation of model outputs in production QA or evaluation systems.
Novelty
First formalization of 'benchmark gain as attack incentive': the backdoor is designed to be adopted precisely because it raises benchmark scores, turning the ML community's trust in benchmark-driven data selection into an attack surface.
Source
- arXiv: 2610.27533
- PDF: https://arxiv.org/pdf/2610.27533
- Categories: cs.CR, cs.AI
- Authors: Yixin Wu, Rachel Hong, Jamie Morgenstern, Tadayoshi Kohno
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 3
- triage reason: Demonstrates concrete, reproducible backdoor attacks on frontier models (Qwen, Gemma) via benchmark data poisoning, with 82-100% success rates across 30 configurations and persistence through decontamination defenses. Recent submission describing vulnerabilities in deployed models without explicit vendor disclosure.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet