SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-375]·STATUS:ACTIVE·OPENED:2026-11-07·UPDATED:2026-11-07

FOL gradient-norm signal identifies embedding states where safety labels flip under tiny perturbations (BFR up to 46%)

submitted_by:@mexiQQ
alignmentfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2611.07530 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.73, flags: [no-prompt-excerpt])

Category

alignment

Model

Qwen2.5-14B-Instruct

Surface

API (white-box embedding-space optimization + local perturbation diagnostic)

Setup

JailBound's Safety-Sensitivity Branch maximizes 𝒪⁺(e) = ℒ_atk(e) + λ·FOL(e), where FOL(e) = ε·‖∇_e ℒ_atk(e)‖_F is the scaled gradient norm of the attack objective. This drives search toward embedding states with high local first-order variation. After optimization (T=200 gradient-ascent steps via Adam), the resulting states are perturbed at radii ρ ∈ {0.05, 0.40, 3.20, 6.40} and re-evaluated by the external judge; Behavioral Flip Rate (BFR) measures the fraction of perturbations that flip the judge's safe/unsafe label. The formal BFR definition and full radius sweep are in Appendix A.1 (not included in body).

Reproduction anchor

none — no public code URL in body. Table 2 in the main text provides BFR values at four radii for both branches on JailBound and S-Eval sources. Appendix A.1 contains the formal BFR definition and complete radius sweep.

Observed behavior

Safety-Sensitivity states achieve BFR of 14.3% / 17.9% / 34.0% / 39.3% at ρ = 0.05 / 0.40 / 3.20 / 6.40 on JailBound, and 21.4% / 25.0% / 41.1% / 46.4% on S-Eval — consistently higher than High-Value states at every radius (High-Value peaks at 12.5% / 16.1%). The Safety-Sensitivity Branch also achieves higher transfer success rates than the High-Value Branch across all nine transfer targets in Table 5.

Expected behavior

Safety decisions of an aligned model should be robust to small local perturbations in embedding space; a well-calibrated model should not have identifiable embedding regions where tiny changes (ρ = 0.05) flip its safety judgments nearly 1 in 7 times.

Reproducibility

medium

Threat model

An attacker with white-box access to any model in the same family as the optimization surrogate can locate locally unstable embedding states that act as reliable jailbreak seeds: small random or gradient-guided perturbations of these states frequently cross the safety boundary, enabling cheap black-box transfer attacks against downstream deployments of related models.

Novelty

First explicit demonstration that gradient-norm magnitude in embedding space (FOL) is a practical proxy for safety-label instability: maximizing FOL reliably finds states with anomalously high BFR, and these same states exhibit higher transferability than states that maximize attack success directly — suggesting local sensitivity and transferability are linked properties not captured by ASR.

Source

Triage notes (auto)

  • paperType: benchmark
  • estimatedCaseCount: 3
  • triage reason: Presents JailBound, a systematic jailbreak benchmark evaluating 46 LLMs across synthesized attack prompts organized by threat taxonomy. Demonstrates concrete model-level failures with reproducible setup and quantified attack success rates.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet