Safety-prompted GPT-5.6-Sol-SP blocks harmful actions but worsens multimodal deception recognition
Auto-published from arXiv:2611.08281 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.85, flags: [possible-zero-day, no-prompt-excerpt])
Category
alignment
Model
GPT-5.6-Sol-SP (GPT-5.6-Sol with five explicit safety rules added to system prompt)
Surface
agent harness (Multimodal-SEA, same 608 task evaluation as baseline)
Setup
Five explicit safety rules are appended to GPT-5.6-Sol's system prompt (exact rules in Appendix H.3, not reproduced in the body). The model is then evaluated on the full Multimodal-SEA benchmark under the same multimodal social engineering attacks used for the baseline.
Reproduction anchor
code: https://github.com/Zhitao-He/M-SEABench; safety prompt rules verbatim in Appendix H.3
Observed behavior
Chinese-task attack success rate drops from 34.8% to 10.8%, and English-task rate drops from 34.2% to 14.0% — but Multimodal Deception Miss Rate increases from 50.4% to 63.5% (Chinese) and from 70.9% to 78.8% (English). The model refuses more harmful actions while simultaneously failing to flag more forged content as deceptive.
Expected behavior
Safety instructions should prevent harmful actions while at minimum maintaining — and ideally improving — the agent's ability to explicitly identify forged documents, screenshots, and videos as deceptive.
Reproducibility
medium
Threat model
Organizations deploying personal AI agents with hardened system prompts may believe their agents are substantially protected. The reduced attack success rate provides false assurance: the agent still fails to recognize the majority of deceptive content, leaving users uninformed about active fraud attempts and potentially enabling escalation through other channels.
Novelty
Quantified demonstration of a safety/recognition trade-off: behavioral action constraints reduce attack success without improving (and measurably degrading) deception recognition, suggesting that instruction-level defenses and perceptual deception recognition require separate mechanisms.
Source
- arXiv: 2611.08281
- PDF: https://arxiv.org/pdf/2611.08281
- Categories: cs.CL
- Authors: Zhitao He, Chi Him Zhang, Chengru Han, Yi R Fung
Triage notes (auto)
- paperType:
benchmark - estimatedCaseCount: 2
- triage reason: Multimodal-SEA is a comprehensive red-team benchmark with 608 expert-designed tasks across 152 attack scenarios demonstrating concrete vulnerabilities in 7 production multimodal models (7.1%–34.5% attack success rates). No explicit vendor disclosure mentioned.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet