Fictional Scenario Attack bypasses code-generation guardrails at ~100% ASR by embedding malware intent in legitimate dev narratives

Auto-published from arXiv:2609.09798 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.77, flags: [no-prompt-excerpt, possible-zero-day]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

jailbreak

Model

LlamaGuard4-v2-24B, NemoGuard, WildGuard, MDJudge, Qwen3Guard; underlying LLMs: CodeLlama-13B-Instruct, DeepSeekCoder-V2-Lite-16B, Qwen3-30B, GPT5-mini

Surface

API / code-generation pipeline with guardrail wrapper

Setup

The Fictional Scenario Attack (FSA) constructs prompts in three steps: (1) extract the functional behavior from a malware TTP prompt, replacing hostile terminology with neutral technical language; (2) generate a plausible legitimate application scenario (name, usage context, behavioral justification) that legitimately requires those same capabilities; (3) merge the base malicious objective with the fictional context into a single prompt. 1000 FSA prompts were generated; a 300-prompt subset is used for evaluation. The paper provides system prompts for steps 1 and 2 in the appendix but does not publish a verbatim final FSA prompt.

Reproduction anchor

benchmark: CS-Guard FSA subset (n=300); data stated to be released under CC BY-NC-SA 4.0 but no repository URL is present in the paper

Observed behavior

Internal-type guardrails allow 85.9%-99.5% of FSA prompts through. Input classifier guardrails (LlamaGuard4, NemoGuard, WildGuard) pass 73.3%-98.5% of FSA prompts undetected. Strategy-type guardrails (SmoothLLM, PAT, RPO) reduce ASR modestly; best (PAT) leaves 64.7% ASR. Multi-turn FSA variant achieves 65.0%-92.3% ASR across all guardrails.

Expected behavior

Guardrails should detect and refuse requests for malware generation regardless of fictional framing; functional malware capabilities should be refused even when the stated narrative context is legitimate-sounding.

Reproducibility

medium

Threat model

An adversary using a commercially deployed coding assistant guarded by LlamaGuard4 or NemoGuard can craft FSA-style prompts (automatable via LLM) to bypass the guardrail at near-100% rate and obtain functional malware code.

Novelty

First systematic demonstration that embedding malicious coding intent inside a plausible fictional software-development narrative defeats virtually all nine tested guardrail architectures at ASR approaching 100%.

Source

• arXiv: 2609.09798
• PDF: https://arxiv.org/pdf/2609.09798
• Categories: cs.CR, cs.AI
• Authors: Jinyang Li, Mingyu Guo, Hung X. Nguyen

Triage notes (auto)

• paperType: benchmark
• estimatedCaseCount: 3
• triage reason: Systematic benchmark of LLM code-generation vulnerabilities with 1000+ malware prompts, 7 jailbreak techniques, and a novel fictional-scenario attack (FSA); demonstrates ~50–100% attack success rates across 9 guardrails and 7 LLMs.

─────────────────────────────────────────────────────────────────────