Benign unlearning requests unintentionally erase sensitive entities sharing internal representations with the requested entity
Auto-published from arXiv:2611.15013 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.73, flags: [no-prompt-excerpt, possible-zero-day])
Category
alignment
Model
LLaMA-3.1-8B-Instruct; Qwen-2.5-3B-Instruct; Qwen-2.5-7B-Instruct; Qwen-2.5-14B-Instruct
Surface
LLM provider unlearning API (entity-level removal interface)
Setup
A legitimate user submits an unlearning request for an entity e (e.g., a person or brand they are entitled to have removed). Unknown to both user and provider, e is superposed in the model's parameter subspace with a sensitive entity T (e.g., a medical term, demographic group description, or critical factual entity). The provider executes the standard Gradient Ascent or NPO unlearning on e. No adversarial intent is required; the collision arises from the geometry of the model's representations. Probe templates are the same three QA forms as the deliberate attack (Section VII-A); no verbatim template quoted in body.
Reproduction anchor
Same experimental infrastructure as the deliberate PUA: CounterFact/UMLS/SBIC target sets, Wikidata 5M removable pool, KSD Algorithm 1 for detecting incidental superposition, GA and NPO unlearning backends. The unintentional-collision scenario is described in the introduction and threat model (Section IV); separate ablation results for this sub-case are in the experiments (Table I, R-group and T-group columns) but the full table is cut off in the available body excerpt.
Observed behavior
Knowledge about T, a sensitive entity never requested for removal, degrades after a benign legitimate unlearning of a superposed entity. The model may still answer fluently but factually incorrectly (e.g., misstate a medical concept or describe a demographic group inaccurately), because the removal is silent — the model's output is degraded in content while remaining fluent in form.
Expected behavior
Selective unlearning should leave all non-requested entities intact; a provider honoring a legitimate request should have no unintended effect on sensitive entities outside the scope of that request.
Reproducibility
medium
Threat model
LLM providers offering unlearning APIs (for GDPR/copyright compliance) face latent collateral damage even from fully legitimate requests: a routine privacy deletion could silently corrupt medical, legal, or socially sensitive knowledge that the provider needs to preserve, harming downstream users who rely on those facts. The threat is systemic, not attacker-driven, and invisible to both the requester and standard post-unlearning verification.
Novelty
Demonstrates that knowledge superposition converts ordinary, well-intentioned unlearning into an uncontrolled side-channel for silent misinformation, even without any adversary — reframing collateral unlearning damage from a utility/quality issue into a security and reliability risk.
Source
- arXiv: 2611.15013
- PDF: https://arxiv.org/pdf/2611.15013
- Categories: cs.CR
- Authors: Jiayang Li, Tianqing Zhu, Longxiang Gao, Huajie Chen, Dayong Ye, Shang Wang, Huan Huo, Wanlei Zhou
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 2
- triage reason: Introduces Proxy Unlearning Attack (PUA), a concrete vulnerability where legitimate unlearning requests exploit knowledge superposition to remove unintended sensitive entities from LLMs. Attack is reproducible, model-level, and no vendor disclosure is mentioned.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet