Backdoored DINOv2/CLIP encoder achieves 40–71% tumor misclassification flip rate on Camelyon17 pathology after clean hospital finetuning
Auto-published from arXiv:2611.14970 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.91, no flags)
Category
backdoor-attack
Model
DINOv2, CLIP (vit_base_patch16_clip_224.openai_ft_in12k_in1k), ResNet-50 (IMAGENET1K_V2)
Surface
medical image classification — whole-slide pathology (Camelyon17 breast cancer lymph node slides)
Setup
Attacker targets class 'no tumor' (class 0) in Camelyon17, reasoning that underdiagnosis delays screening and treatment. Attacker collects a small public surrogate set from Camelyon17 hospitals 0–2 (task-aligned, sample-disjoint from victim data). The poisoned encoder is released; the victim finetunes on hospitals 0–2 and deploys on hospital 3 (unseen distribution). The trigger is an optimized pixel-space perturbation applied at inference time. Full setup in Section 4 (Models/Datasets) and Appendix B.
Reproduction anchor
code: https://github.com/QixuanJin99/poison-then-hide; dataset: Camelyon17 via WILDS benchmark (Koh et al. 2021); per-model results in Table 1 (Section 5.2)
Observed behavior
After victim's clean finetuning, DINOv2 achieves 71.54% original ASR and 40.47% Flip-ASR on hospital 3 (benign accuracy 99.32%); CLIP achieves 73.32% ASR and 42.16% Flip-ASR (benign accuracy 94.23%); ResNet-50 achieves 91.94% ASR and 71.81% Flip-ASR (benign accuracy 87.95%). Flip-ASR margins over the strongest non-PtH baseline are 17.32 pp (DINOv2/C17) and 8.55–17.33 pp (ResNet50/CLIP). The attack transfers across the hospital distribution shift without any attacker access to hospital-3 data.
Expected behavior
A correctly behaving diagnostic model should correctly classify tumor-present slides even when inputs carry the trigger perturbation; backdoor-free transfer learning should not introduce systematic false-negative bias.
Reproducibility
high
Threat model
A malicious encoder supplier (e.g., a compromised model hub contributor) targets medical AI developers who finetune foundation vision encoders for cancer screening. Clinics or research hospitals that adopt the poisoned encoder will deploy a system that produces false-negative diagnoses (tumor classified as 'no tumor') for any patient whose slide is perturbed with the trigger, potentially delaying treatment across all institutions using the shared base model.
Novelty
Provides the first concrete demonstration that a finetuning-activated backdoor targeting underdiagnosis transfers across realistic hospital distribution shifts in clinical pathology, with high flip rates maintained even under full-model finetuning on entirely clean hospital data.
Source
- arXiv: 2611.14970
- PDF: https://arxiv.org/pdf/2611.14970
- Categories: cs.LG, cs.CR, cs.CV
- Authors: Qixuan Jin, Abinitha Gourabathina, Vinith Menon Suriyakumar, Walter Gerych, Marzyeh Ghassemi
Triage notes (auto)
- paperType:
red-team-vuln - estimatedCaseCount: 2
- triage reason: Demonstrates concrete, reproducible backdoor attacks on vision encoders with 97% success rates and shows that standard defenses fail to remove implanted backdoors, providing multiple model-level failure cases (dormant backdoor activation during finetuning, persistence through defense mechanisms, reactivation after benign finetuning).
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet