SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-324]·STATUS:ACTIVE·OPENED:2026-10-27·UPDATED:2026-10-27

All six frontier models show uniform jailbreak vulnerability across 12 risk categories including Chem/Bio and Cybercrime without attacker taxonomy knowledge

submitted_by:@mexiQQ
alignmentneeds-disclosure-reviewfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2610.31937 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.78, flags: [no-prompt-excerpt]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

alignment

Model

Claude 3.5 Sonnet, Claude 3 Haiku, GPT-4o, GPT-4o mini, Llama3-70B-Instruct, Llama3-8B-Instruct

Surface

API (black-box)

Setup

M³-HC (Mistral-Large-2402 attacker/state LLM) evaluated across a merged 12-category risk space: Hate, Illegal, Misinfo, Porn, Profanity, Self-Harm, Sexism, Violence (from Crescendo Tasks) plus Chem/Bio, Cybercrime, Harassment, Harmful (from HarmBench). The attacker receives no predefined risk taxonomy — it generates adversarial prompts solely from the compact state representation and the task description. Uniformity measured by Gini index over ASR@5 distributions across all 12 categories (lower = more uniform coverage). Category-level breakdown in Appendix A.7 (not in extracted body).

Reproduction anchor

Benchmark: Crescendo Tasks (15 tasks, 8 categories) + HarmBench 90-task subset (6 categories), fully described in Section 4; Gini index methodology in Section 5.3; per-category ASR details in Appendix A.7 (referenced, not shown); Algorithm 1 in Appendix A.2 (included); no code repo URL found.

Observed behavior

M³-HC achieves Gini index of 4.24–6.22 across all six target models, versus 6.20–10.76 for all baselines — up to 4.82 points lower, indicating substantially more uniform attack success across categories. Even traditionally difficult categories (e.g., Hate Speech) are exploited more consistently than by any baseline. Paper notes Violence, Self-Harm, and Profanity are easiest; Hate remains hardest but is still more uniformly attacked by M³.

Expected behavior

Safety-aligned models should refuse harmful requests uniformly across all risk categories; the Gini index of attack success should be irrelevant because ASR should remain near 0% across all categories.

Reproducibility

medium

Threat model

Adversaries need no domain-specific knowledge or per-category attack tuning to exploit frontier models across high-stakes harm categories including chemical/biological instructions and cybercrime. Safety evaluations that test only single categories or rely on category-specific alignment fine-tuning will underestimate true multi-turn vulnerability breadth for any deployed LLM.

Novelty

First demonstration that a strategy-agnostic, taxonomy-free MDP attacker achieves near-uniform coverage across 12 heterogeneous risk categories, showing that per-category safety alignment does not compose into robust multi-turn resistance.

Source

Triage notes (auto)

  • paperType: red-team-vuln
  • estimatedCaseCount: 2
  • triage reason: Automated red-teaming framework demonstrating concrete multi-turn jailbreak successes against six target models with quantified ASR metrics; inherently case-rich for reproducible model-level failures.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet