Proxy Unlearning Attack silently degrades target entities never named in any unlearning request via knowledge superposition
Auto-published from arXiv:2611.15013 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [no-prompt-excerpt, possible-zero-day])
Category
other
Model
LLaMA-3.1-8B-Instruct; Qwen-2.5-3B-Instruct; Qwen-2.5-7B-Instruct; Qwen-2.5-14B-Instruct
Surface
LLM provider unlearning API (entity-level removal interface)
Setup
Adversary holds a set of sensitive target entities T (drawn from CounterFact, UMLS, or SBIC) it wants to degrade and a removable set U (Wikidata 5M entities, |U|=100K in main experiments). Using Algorithm 1 (KSD) with whitened cosine threshold τ=0.9, the adversary identifies proxy entities S ⊆ U whose internal representations overlap with T in at least one transformer layer. It submits S as an ordinary unlearning request. The provider executes the request faithfully (using Gradient Ascent or NPO), and standard verification confirms only that S was removed. No verbatim prompt template is quoted; probing uses three question forms per entity: 'What is X?', 'What category or type does X belong to?', 'What is X associated with?' (Section VII-A).
Reproduction anchor
No code artifact URL visible in body text. Benchmarks: CounterFact (general facts), UMLS (biomedical), SBIC (social bias) as target entity sources; Wikidata 5M as removable pool. Full KSD algorithm in Algorithm 1 (body). Unlearning backends: Gradient Ascent and NPO. Decoding/probing details in Section VII-A. Black-box results in Section VII-B-2 (not in available body excerpt).
Observed behavior
After unlearning the proxy set S, the model's LLM-judge factual score and Ref-NLL for target entities T — which were never named in any request — degrade substantially across all three domains and both model families, while the retain group R (unrelated entities) stays largely intact, confirming the damage is targeted rather than broad. Standard provider verification reports successful removal of S with no indication that T was harmed.
Expected behavior
Unlearning should be selective: only the requested entities S lose knowledge; entities in T that were never submitted for removal should remain fully intact.
Reproducibility
medium
Threat model
An adversary with access to a model's (or surrogate's) internals submits legitimate unlearning requests to an LLM provider's removal API, covertly erasing a competitor's brand knowledge, degrading medical concepts (UMLS), or introducing social bias (SBIC) about demographic groups — all without ever naming the actual targets. The provider's own faithful compliance becomes the attack vector; standard verification cannot detect the harm.
Novelty
First formalization that the legitimate LLM unlearning interface is itself an attack surface: no training-data access, no backdoor planting, and no model comparison needed — the provider's correct execution of an ordinary request causes targeted, silent harm to entities outside the request.
Source
- arXiv: 2611.15013
- PDF: https://arxiv.org/pdf/2611.15013
- Categories: cs.CR
- Authors: Jiayang Li, Tianqing Zhu, Longxiang Gao, Huajie Chen, Dayong Ye, Shang Wang, Huan Huo, Wanlei Zhou
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 2
- triage reason: Introduces Proxy Unlearning Attack (PUA), a concrete vulnerability where legitimate unlearning requests exploit knowledge superposition to remove unintended sensitive entities from LLMs. Attack is reproducible, model-level, and no vendor disclosure is mentioned.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet