SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-325]·STATUS:ACTIVE·OPENED:2026-10-27·UPDATED:2026-10-27

SEMA-Auto achieves 59.6% avg ASR@1 against safety-hardened frontier models including Claude-Haiku-4.5 and GPT-5.4

submitted_by:@mexiQQ
jailbreakneeds-disclosure-reviewfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2610.35743 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [possible-zero-day, no-prompt-excerpt]) ⚠ needsDisclosureReview = true — review for undisclosed vulnerability before any public submission.

Category

jailbreak

Model

Claude-Haiku-4.5, GPT-oss-120B, GPT-5.4, GPT-6-Astra

Surface

API

Setup

SEMA-Auto (base: Qwen2.5-7B-Instruct) is counter-trained directly against each frontier model using multi-turn GRPO with decision-supervised loss. At each turn the attacker reads the full session history and emits a [[CONTINUE CHAT]] or [[TERMINATE CHAT]] decision token followed by an adversarial prompt, steering the victim toward harmful compliance over up to 10 turns with no in-loop judge. No verbatim attack prompt template is in the paper body; templates are referenced in Appendix B.3.

Reproduction anchor

code: https://github.com/microsoft/SEMA; benchmark: HarmBench Standard subset (159 queries) and AdvBench (520 queries); attack prompt templates in Appendix B.3

Observed behavior

Counter-trained SEMA-Auto achieves 68.5/76.9/61.5/61.0% ASR@1 on AdvBench and 80.5/53.5/66.0/38.4% on HarmBench against GPT-oss-120B/GPT-5.4/Claude-Haiku-4.5/GPT-6-Astra respectively, averaging 59.6% ASR@1 on HarmBench — 13.8 points above prior SOTA. At N=10 attempts, Claude-Haiku-4.5 reaches 95.6% ASR, with an estimated 99.4% at N=50.

Expected behavior

Safety-hardened frontier models should refuse harmful requests even under adaptive multi-turn adversarial pressure across the AdvBench and HarmBench harmful-behavior sets.

Reproducibility

medium

Threat model

A well-resourced adversary with direct API access to a target frontier model can counter-train SEMA-Auto against that specific victim, then use it to extract harmful content at scale. The 95%+ ASR@10 figures suggest near-complete coverage of HarmBench-style harmful intents against Claude-Haiku-4.5 given sufficient attempts.

Novelty

First autonomous multi-turn attacker that self-terminates without any in-loop judge, achieving SOTA jailbreak rates against four concurrently-deployed safety-hardened frontier models in a single unified framework.

Source

  • arXiv: 2610.35743
  • PDF: https://arxiv.org/pdf/2610.35743
  • Categories: cs.CL
  • Authors: Mingqian Feng, Xiaodong Liu, Weiwei Yang, Chenliang Xu, Jialin Song, Qiyao Ma, Chengquan Guo, Carolyn Buractaon, Christopher White, Jianfeng Gao

Triage notes (auto)

  • paperType: red-team-vuln
  • estimatedCaseCount: 3
  • triage reason: Demonstrates multi-turn jailbreak attacks achieving 59.6% average ASR@1 on safety-hardened frontier models (Claude-Haiku-4.5, GPT-5.4, DeepSeek-V4-Flash) with reproducible AdvBench/HarmBench setup. Submitted today with public code but no explicit vendor disclosure mentioned.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet