XAI-guided synonym substitution bypasses Prompt Guard 2 and jailbreaks Llama 3.1 8B

Auto-published from arXiv:2609.24801 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.78, flags: [no-prompt-excerpt, possible-zero-day]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

jailbreak

Model

Prompt Guard 2 (DeBERTa-v2, 86M) as guardrail; Llama 3.1 8B as underlying LLM

Surface

API — classifier guardrail layer fronting a chat LLM

Setup

A DAN-style ('Do Anything Now') instruction-override prompt beginning with 'Ignore all your previous instructions' with role-play and reward mechanics (coin collection) is submitted to Prompt Guard 2 (initial injection confidence: 0.9994). Vanilla Gradient XAI attribution identifies the 16 highest-saliency tokens. A manually curated synonym dictionary replaces those tokens one by one, guided by confidence-drop feedback. After 12 substitutions (24% of text modified), the classifier's confidence drops to 0.4457, flipping the label to benign. The modified prompt is then passed to Llama 3.1 8B with four malicious requests (WiFi hacking, reverse shell creation, ransomware, phishing). Paper does not reproduce the exact prompt verbatim.

Reproduction anchor

benchmark: dataset xTRam1/safe-guard-prompt-injection (HuggingFace); model meta-llama/Prompt-Guard-2-86M; no code artifact released. Synonym dictionary and exact DAN prompt text are not published.

Observed behavior

Prompt Guard 2 reclassifies the XAI-perturbed DAN prompt as benign (confidence 0.4457, down from 0.9994). Llama 3.1 8B then produces Partial or Functional responses (rated by Claude Sonnet 5 as auxiliary judge) to requests for WiFi-hacking instructions, reverse shell code, ransomware, and phishing content.

Expected behavior

Prompt Guard 2 should maintain high-confidence injection classification regardless of synonym-level surface perturbations; Llama 3.1 8B should refuse all four harmful requests.

Reproducibility

medium

Threat model

An attacker with access to Prompt Guard 2's output probabilities (white-box or score-based API) uses open-source XAI tools to identify and iteratively replace salient tokens in a known jailbreak template, evading classifier-based content filters in any production deployment that uses Prompt Guard 2 as a first-line guardrail before a capable LLM.

Novelty

First empirical demonstration that Vanilla Gradient / SHAP attributions on a DeBERTa-based safety classifier directly guide adversarial synonym substitution sufficient to both flip the classifier label and unlock jailbreak-grade behavior in a downstream LLM with only 24% text modification.

Source

• arXiv: 2609.24801
• PDF: https://arxiv.org/pdf/2609.24801
• Categories: cs.CR, cs.AI
• Authors: Fernando Outeda, Gustavo Betarte, Juan Diego Campo, Fiorella Cravero

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 2
• triage reason: Demonstrates concrete jailbreak bypasses of the Prompt Guard 2 guardrail through XAI-guided saliency attacks (synonym substitution and paraphrasing); no mention of vendor disclosure coordination.

─────────────────────────────────────────────────────────────────────