OLMo-2-7B SFT leaves F2 empathy-drowns-practicality displacement flat at 52.3%; only DPO suppresses it
Auto-published from arXiv:2610.25522 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [no-prompt-excerpt])
Category
alignment
Model
OLMo-2-7B
Surface
API, single-turn, no system prompt
Setup
GDB F2 (Mixed Empathy-with-Constraints) scenarios: user expresses emotional distress while also facing a practical hard constraint. Matched OLMo-2-7B checkpoints at Base, SFT, and DPO stages evaluated on 65 F2 scenarios. No verbatim prompt quoted in paper body; full scenarios in released dataset. Temperature 0.7, no system prompt.
Reproduction anchor
code: https://github.com/Guylaban/GDB | dataset: https://huggingface.co/datasets/guylaban/gdb-benchmark | post-training analysis: Appendix L
Observed behavior
OLMo-2-7B SFT F2 displacement is 52.3%, identical to the base model. Responses address only the emotional component ('It's okay to feel overwhelmed') while omitting practical constraint guidance. Rate drops to 13.8% only after DPO (p<0.001). Mistral v0.2->v0.3 and Qwen 2.5 Base->Instruct replicate the pattern.
Expected behavior
Integrate empathic acknowledgment with direct practical guidance; validation must not displace substance — both must co-occur.
Reproducibility
high
Threat model
Organizations evaluating whether SFT-based alignment is sufficient for advice-giving deployments (health coaching, financial chatbots, HR tools). Teams relying only on supervised fine-tuning may ship models with latent displacement failures invisible to standard helpfulness evals.
Novelty
First controlled post-training ablation showing F2 goal displacement is a preference-optimization problem — SFT is blind to it — providing a mechanistic handle for alignment practitioners.
Source
- arXiv: 2610.25522
- PDF: https://arxiv.org/pdf/2610.25522
- Categories: cs.AI
- Authors: Dean Ariel, Guy Laban
Triage notes (auto)
- paperType:
benchmark - estimatedCaseCount: 3
- triage reason: Introduces GDB benchmark with 230 reproducible test scenarios measuring goal displacement across 18 models; identifies three distinct failure modes (trap displacement, reverse displacement, mixed empathy-with-constraints) with human-scored annotations and evaluation code.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet