Utility-hijacking amplifies false-fact injection 7x on Qwen3-32B via scoring-model selector
Auto-published from arXiv:2610.29886 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.88, flags: [possible-zero-day])
Category
hallucination
Model
Qwen3-32B
Surface
API (fine-tuned model via poisoned instruction-tuning data selection)
Setup
The attacker generates false-fact bearing instruction records and scores them with a scoring-model selector (a larger LLM used as a quality judge). Because the scoring model is larger and thus more sensitive to stylistic quality, it preferentially selects well-crafted false-fact records. The generate-and-rerank procedure identifies the highest-scoring false-fact candidate per instruction and submits it; top-k selection then concentrates these records. The false-fact enrichment effect is measured on TruthfulQA-MC1. No verbatim false-fact prompt is quoted in the body; appendix Table S4 contains the selector-payload compatibility matrix.
Reproduction anchor
code: arXiv ancillary file code_and_data_supplement.zip (SHA-256: 2824e80e2fdb591444f067d714b5816edd79eb49422c93d898c1a18a60f10cb3); selector-payload matrix in Appendix Table S4; evaluation configs in configs/experiment_runs.csv
Observed behavior
False-fact enrichment (ratio of poisoned-to-baseline false-fact records in the selected set) scales from 2.84x at smaller scoring-model size to 7.04x at Qwen3-32B scale. The victim model's TruthfulQA-MC1 score degrades by −0.062 (95% CI: [−0.079, −0.045]) after fine-tuning on the poisoned selection.
Expected behavior
The data selector should dilute or eliminate false-fact records; the fine-tuned model should maintain baseline truthfulness on TruthfulQA.
Reproducibility
high
Threat model
A supplier targeting a production fine-tuning pipeline that uses a large LLM as a quality scorer (a common practice for instruction dataset curation) can inject systematically false factual beliefs into the resulting model. Users of deployed models downstream receive confidently wrong answers on factual queries, with no visible sign of tampering in the training pipeline.
Novelty
Demonstrates a counterintuitive scaling law for data-poisoning severity: using a larger, ostensibly more capable quality-scoring model as the selector increases false-fact concentration by up to 7x, inverting the naive expectation that better selectors improve safety.
Source
- arXiv: 2610.29886
- PDF: https://arxiv.org/pdf/2610.29886
- Categories: cs.CR
- Authors: Yahao Ren, Jian Lou, Huadi Zheng, Jiawei Duan
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 3
- triage reason: First systematic characterization of utility-hijacking attacks against instruction-data selectors with concrete, quantified failures (over-refusal 0→94%, insecure code up to 56/90) across three payloads and multiple Qwen models; no explicit disclosure statement visible.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet