Refusal direction transferred from Llama-3.1-8B to Falcon-Mamba-7B enables cross-architecture activation-steering jailbreak

Auto-published from arXiv:2609.04721 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.78, flags: [no-prompt-excerpt, possible-zero-day])

Category

alignment

Model

Falcon-Mamba-7B (target SSM); Llama-3.1-8B (source transformer)

Surface

API with white-box activation steering

Setup

A harm-refusal linear probe is trained on Llama-3.1-8B's residual stream using AdvBench harmful and Alpaca benign prompts. A rigid rotation (orthogonal matrix) aligns Llama's representation space to Falcon-Mamba-7B's. The transported refusal direction is then subtracted from the SSM's residual stream at the write-site layer (mixer output before addition) during inference on harmful prompts. No verbatim code is quoted; the method is described in the main body with appendix detail.

Reproduction anchor

benchmark: AdvBench (harmful_behaviors) + Alpaca (benign); models: Llama-3.1-8B and Falcon-Mamba-7B; methodology in paper body (rigid rotation cross-architecture transport); code to be released upon publication, no URL given

Observed behavior

Removing the transported (Llama-derived) direction from Falcon-Mamba-7B achieves 36.5% attack success rate, compared to 41.5% for the native (Mamba-trained) direction and 9.0% for a random same-size direction. The probe also achieves AUROC 1.00 on held-out harmful/benign classification on the SSM after transport.

Expected behavior

A direction extracted from a completely different architecture (transformer vs. SSM) and mapped over via a simple rotation should not function as a meaningful refusal direction on the target model. The result should be indistinguishable from a random control.

Reproducibility

medium

Threat model

A white-box adversary with access to an SSM's weights can bootstrap an attack using only a pre-trained refusal probe from a more-studied transformer model, requiring no SSM-specific labeled data. This lowers the barrier to activation-steering jailbreaks against newly deployed SSM architectures where safety representations have not yet been publicly characterized.

Novelty

First demonstration that a harm probe trained on a transformer transfers to a structurally dissimilar SSM via a rigid rotation alone, showing safety representations are architecture-agnostic and cross-architecture activation-steering attacks are feasible with minimal per-target effort.

Source

• arXiv: 2609.04721
• PDF: https://arxiv.org/pdf/2609.04721
• Categories: cs.LG
• Authors: Preethi Carmel Bosco, Gopalakrishnan Srinivasan

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 2
• triage reason: Systematically demonstrates how to locate and steer a universal refusal direction across four model architectures (transformers, SSMs, recurrent, hybrid) to cause jailbreaks. Core finding: removing or rotating the identified direction makes models refuse harmful requests they otherwise would, and this transfers across architectures without explicit vendor disclosure mentioned.

─────────────────────────────────────────────────────────────────────