Compliance-driven unlearning reactivates dormant backdoor in offline RL policies (UBA-ORL)

Auto-published from arXiv:2609.22711 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.88, flags: [no-prompt-excerpt, possible-zero-day])

Category

backdoor-attack

Model

TD3+BC, BCQ, IQL trained on D4RL Hopper-medium-expert and Walker2d-medium datasets

Surface

Offline RL training pipeline with trajectory-level unlearning (data deletion) API

Setup

Attacker injects two trajectory subsets into the offline dataset: (1) backdoor (BD) trajectories linking a high-quantile velocity-dimension trigger pattern (indices 5–10 for Hopper, 8–13 for Walker2d, applied as s̃ = (1−m)⊙s + m⊙τ) to malicious actions under inflated rewards, and (2) camouflage (CM) trajectories sharing the same trigger but preserving benign actions with equally high rewards. Poisoning budget is ≤10% total (15% for Hopper/BCQ). During training BD and CM compete, suppressing trigger response. The attacker then submits a legitimate GDPR/compliance deletion request for only the CM subset, leaving BD unchallenged. No verbatim prompt excerpt exists (RL setting); the trigger construction formula and D4RL dataset identifiers fully specify reproduction.

Reproduction anchor

code: https://github.com/Cormac315/UBA-ORL — includes D4RL environments (Hopper-medium-expert, Walker2d-medium), MuJoCo 2.1.0, D4RL 1.1, d3rlpy 1.0.0, PyTorch 2.10.0; trigger schedule L10 (single 10-step window) specified in paper body

Observed behavior

Across all 6 algorithm×environment configurations, Performance Drop (PD) under the trigger increases substantially after CM deletion: Hopper/TD3+BC +20.9 pp (4.8%→25.7%), Hopper/BCQ +26.0 pp (19.2%→45.2%), Hopper/IQL +13.3 pp (26.0%→39.3%), Walker2d/TD3+BC +21.2 pp (4.0%→25.2%), Walker2d/BCQ +39.2 pp (−6.2%→33.1%), Walker2d/IQL +19.6 pp (12.7%→32.3%). Average activation gap +23.4 pp. Before CM deletion, Walker2d/BCQ shows −6.2% PD (trigger slightly improves return), providing near-perfect stealth.

Expected behavior

After honoring a legitimate deletion request, the trained policy should be no more susceptible to the trigger than a policy trained without the deleted data; PD should remain at or below pre-deletion baselines.

Reproducibility

high

Threat model

A malicious data contributor to a compliance-compliant offline RL platform (e.g., a robotics or autonomous-driving data marketplace) injects dual trajectories, then submits a GDPR/right-to-erasure deletion request for only the camouflage subset. The platform's unlearning service — intended to protect user privacy — becomes the activation mechanism for a hidden backdoor, causing the deployed policy to take adversarially chosen actions (e.g., unsafe maneuvers) whenever the trigger state pattern is present. Affected parties: operators of offline RL systems that expose batch-trajectory deletion APIs, and end-users of downstream safety-critical applications.

Novelty

First demonstrated attack in which the compliance-driven unlearning procedure itself is the activation trigger — transforming privacy-protection infrastructure into an on-demand backdoor switch, an entirely new attack surface not previously identified in the offline RL or machine unlearning literature.

Source

• arXiv: 2609.22711
• PDF: https://arxiv.org/pdf/2609.22711
• Categories: cs.CR
• Authors: Fengyi Wang, Cong Li, Lulu Xue, Qiyu Leng, Ziqi Zhou, Peijin Guo

Triage notes (auto)

• paperType: red-team-vuln
• estimatedCaseCount: 1
• triage reason: First published backdoor attack against offline RL via compliance-driven unlearning. Demonstrates novel, reproducible vulnerability affecting increasingly-deployed systems in autonomous driving and robotic control, with no disclosed vendor coordination.

─────────────────────────────────────────────────────────────────────