SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-287]·STATUS:ACTIVE·OPENED:2026-10-21·UPDATED:2026-10-21

BenchDoor backdoor achieves ≥90.33% ASR after ROUGE-L decontamination of benchmark-derived training data

submitted_by:@mexiQQ
from-arxivauto-publisheddata-poisoning
cat case_body.md

Auto-published from arXiv:2610.27533 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.78, flags: [possible-zero-day, no-prompt-excerpt])

Category

data-poisoning

Model

Qwen2.5-3B-Instruct, Gemma-3-4B-IT

Surface

Fine-tuning pipeline with decontamination preprocessing step

Setup

Attack scenario S₂^Decon: the adversary releases synthetic paraphrases of poisoned MMLU or TweetEval test examples. A defender applies standard ROUGE-L decontamination (token-level, case-folded regex tokenization, threshold ≥ 0.7) to filter training data before fine-tuning. The backdoor trigger (parenthesized MMLU labels or AI-topic text for TweetEval sentiment) is preserved in the paraphrases because semantic content survives paraphrase while surface-level n-gram overlap drops below the filter threshold.

Reproduction anchor

benchmark: MMLU and TweetEval Sentiment (same splits as main experiments); decontamination method described in Section 3/4 (ROUGE-L ≥ 0.7, token-level); Table 1 reports S₂^Decon ASRs. No public code URL; authors restrict code to research access.

Observed behavior

After ROUGE-L decontamination, the backdoor survives with attack success rates of at least 90.33% across evaluated configurations. Benchmark performance gains are maintained, so the poisoned dataset still appears beneficial to adopters who apply decontamination as a defense.

Expected behavior

Decontamination should remove or substantially degrade backdoor effectiveness by filtering training examples that are semantically similar to test data; ASR should drop to near-random chance.

Reproducibility

medium

Threat model

ML practitioners who apply ROUGE-L decontamination as a standard hygiene step before fine-tuning on community datasets assume this defends against test-set leakage and poisoning. BenchDoor^Decon demonstrates that this widely-used defense is insufficient, leaving any organization using benchmark-driven data curation vulnerable to backdoored models in production.

Novelty

First demonstration that benchmark-targeted backdoors can be designed to survive standard ROUGE-L decontamination while retaining both benchmark gains and high attack success rates, invalidating a common defense assumption.

Source

Triage notes (auto)

  • paperType: red-team-vuln
  • estimatedCaseCount: 3
  • triage reason: Demonstrates concrete, reproducible backdoor attacks on frontier models (Qwen, Gemma) via benchmark data poisoning, with 82-100% success rates across 30 configurations and persistence through decontamination defenses. Recent submission describing vulnerabilities in deployed models without explicit vendor disclosure.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet