Paraphrase-induced hallucination in Qwen2.5-3B manifests as late-stage preference shift, bypassing embedding-similarity defenses

Auto-published from arXiv:2609.21227 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [no-prompt-excerpt])

Category

hallucination

Model

Qwen2.5-3B

Surface

API

Setup

Diagnostic probing on SimpleQuestions: robustness-failing paraphrase pairs are compared to robustness-preserving pairs via layer-wise embedding distance analysis (question-answer concatenation distance dQA). Standard zero-shot factual QA prompts are used. No verbatim prompt excerpt is provided for this diagnostic; the paper describes inference-time probing of model internals.

Reproduction anchor

code: https://github.com/yuwenhan07/Hallucination-R1

Observed behavior

Robustness-failing cases show systematically larger dQA at final layers but similar intermediate representations. Layer-wise analysis confirms failure emerges as a late-stage output preference shift: the model processes semantically equivalent paraphrases similarly in early layers but diverges in the final answer distribution.

Expected behavior

A robust model should maintain consistent answer distributions across semantically equivalent inputs at all layers, not just early ones.

Reproducibility

medium

Threat model

Safety engineers relying on intermediate-layer embedding similarity checks — a common consistency defense — will systematically miss paraphrase-induced hallucination risk, leaving deployed knowledge-intensive systems silently vulnerable.

Novelty

Mechanistically locates paraphrase-induced hallucination to the late decoding stage rather than early encoding, showing that representation-similarity defenses are architecturally mismatched to this failure class and distinguishing it from standard adversarial input perturbations.

Source

• arXiv: 2609.21227
• PDF: https://arxiv.org/pdf/2609.21227
• Categories: cs.CL, cs.AI
• Authors: Wenhan Yu, Wenxin Wu, Hao Wang, Lei Sha

Triage notes (auto)

• paperType: benchmark
• estimatedCaseCount: 2
• triage reason: Systematic benchmark of a specific, reproducible failure mode: QA models answering factually correctly on original questions but failing on semantically equivalent paraphrases. Tested across multiple standard datasets and model families.

─────────────────────────────────────────────────────────────────────