Prompt Guard 2 is more easily bypassed in Spanish: XAI-guided perturbation needs only 7 steps vs. 12 in English

Auto-published from arXiv:2609.24801 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.77, flags: [possible-zero-day, no-prompt-excerpt]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

jailbreak

Model

Prompt Guard 2 (DeBERTa-v2, 86M) as guardrail; Llama 3.1 8B as underlying LLM

Surface

API — classifier guardrail layer fronting a chat LLM

Setup

The same XAI-guided synonym-substitution protocol is applied to a Spanish-language DAN-style prompt with identical adversarial intent. Vanilla Gradient attributions identify the highest-saliency Spanish tokens; a curated synonym dictionary replaces them in order of saliency while monitoring Prompt Guard 2's output confidence for a label flip. Paper does not reproduce the exact Spanish prompt verbatim.

Reproduction anchor

benchmark: xTRam1/safe-guard-prompt-injection (HuggingFace); model meta-llama/Prompt-Guard-2-86M; procedure mirrors the English experiment described in the paper body; no code artifact released.

Observed behavior

Label flip is achieved after only 7 synonym substitutions (18% text modification), compared to 12 substitutions (24%) required for the English prompt. The paper attributes this gap to Prompt Guard 2 being less robustly calibrated for under-represented languages in its training data.

Expected behavior

Prompt Guard 2 should require at least as many perturbations to bypass in non-English languages as in English; ideally language should not affect attack difficulty.

Reproducibility

medium

Threat model

Attackers targeting multilingual production deployments (e.g., customer-facing chatbots in Spanish-speaking markets) face a lower evasion cost when crafting adversarial prompts in languages under-represented in the guardrail's training distribution, enabling cheaper automated jailbreak campaigns.

Novelty

Empirically quantifies that classifier-based guardrails degrade faster under XAI-guided perturbation in non-English languages, providing a concrete multilingual attack surface that is absent from prior Prompt Guard 2 evaluations.

Source

• arXiv: 2609.24801
• PDF: https://arxiv.org/pdf/2609.24801
• Categories: cs.CR, cs.AI
• Authors: Fernando Outeda, Gustavo Betarte, Juan Diego Campo, Fiorella Cravero

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 2
• triage reason: Demonstrates concrete jailbreak bypasses of the Prompt Guard 2 guardrail through XAI-guided saliency attacks (synonym substitution and paraphrasing); no mention of vendor disclosure coordination.

─────────────────────────────────────────────────────────────────────