Utility-hijacking injects insecure code (CWE-078/295/502) into 56/90 outputs via IFD selector
Auto-published from arXiv:2610.29886 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [possible-zero-day, no-prompt-excerpt])
Category
data-poisoning
Model
Qwen3-32B
Surface
API (fine-tuned model via poisoned instruction-tuning data selection)
Setup
The attacker crafts code-generation instruction records containing vulnerabilities from CWE-078 (OS Command Injection), CWE-295 (Improper Certificate Validation), and CWE-502 (Deserialization of Untrusted Data). These records are shaped via generate-and-rerank to score highly on the IFD selector while retaining the insecure code payload. After top-k selection concentrates them, the victim model is fine-tuned. Without the CONCERT defense, CONCERT reports poison concentration of 6.67x; the downstream insecure-code metric counts unsafe outputs across 90 code-generation test prompts. No verbatim code template is quoted in the body; candidate generation script is at code/over_refusal/scripts/generate_amrs_candidates.py (shared pipeline).
Reproduction anchor
code: arXiv ancillary file code_and_data_supplement.zip (SHA-256: 2824e80e2fdb591444f067d714b5816edd79eb49422c93d898c1a18a60f10cb3); evaluation at code/evaluation/; selection results across K values in ancillary CSVs; CONCERT defense baseline in code/selection/defense_b_fixed_budget_consensus.py
Observed behavior
After fine-tuning on a utility-hijacked selection, Qwen3-32B produces insecure code in 56 out of 90 test prompts (vs. 0/90 on clean baseline). Poison concentration ratio reaches 6.67x. The CONCERT defense reverses this to 0x concentration and 0/90 insecure outputs (100% harm reduction).
Expected behavior
The data selector should reject or dilute vulnerability-bearing code records; the fine-tuned model should generate secure code patterns across standard CWE test cases.
Reproducibility
high
Threat model
A malicious contributor to an open instruction-tuning dataset (e.g., for a coding assistant) submits high-quality-appearing but vulnerability-laden code examples that pass automated quality filters. Organizations deploying the resulting fine-tuned model as a coding assistant ship insecure code to developers at scale, with CWE-class vulnerabilities silently embedded in generated outputs.
Novelty
Provides the first end-to-end demonstration that supplier-side adaptive attacks can reliably smuggle specific CWE-class vulnerabilities through quality-based data selectors, and quantifies both the attack severity (56/90) and the effectiveness of the proposed CONCERT consensus defense (0/90).
Source
- arXiv: 2610.29886
- PDF: https://arxiv.org/pdf/2610.29886
- Categories: cs.CR
- Authors: Yahao Ren, Jian Lou, Huadi Zheng, Jiawei Duan
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 3
- triage reason: First systematic characterization of utility-hijacking attacks against instruction-data selectors with concrete, quantified failures (over-refusal 0→94%, insecure code up to 56/90) across three payloads and multiple Qwen models; no explicit disclosure statement visible.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet