SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-278]·STATUS:ACTIVE·OPENED:2026-10-20·UPDATED:2026-10-20

OLMo-2-7B SFT leaves F2 empathy-drowns-practicality displacement flat at 52.3%; only DPO suppresses it

submitted_by:@mexiQQ
alignmentfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2610.25522 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.82, flags: [no-prompt-excerpt])

Category

alignment

Model

OLMo-2-7B

Surface

API, single-turn, no system prompt

Setup

GDB F2 (Mixed Empathy-with-Constraints) scenarios: user expresses emotional distress while also facing a practical hard constraint. Matched OLMo-2-7B checkpoints at Base, SFT, and DPO stages evaluated on 65 F2 scenarios. No verbatim prompt quoted in paper body; full scenarios in released dataset. Temperature 0.7, no system prompt.

Reproduction anchor

code: https://github.com/Guylaban/GDB | dataset: https://huggingface.co/datasets/guylaban/gdb-benchmark | post-training analysis: Appendix L

Observed behavior

OLMo-2-7B SFT F2 displacement is 52.3%, identical to the base model. Responses address only the emotional component ('It's okay to feel overwhelmed') while omitting practical constraint guidance. Rate drops to 13.8% only after DPO (p<0.001). Mistral v0.2->v0.3 and Qwen 2.5 Base->Instruct replicate the pattern.

Expected behavior

Integrate empathic acknowledgment with direct practical guidance; validation must not displace substance — both must co-occur.

Reproducibility

high

Threat model

Organizations evaluating whether SFT-based alignment is sufficient for advice-giving deployments (health coaching, financial chatbots, HR tools). Teams relying only on supervised fine-tuning may ship models with latent displacement failures invisible to standard helpfulness evals.

Novelty

First controlled post-training ablation showing F2 goal displacement is a preference-optimization problem — SFT is blind to it — providing a mechanistic handle for alignment practitioners.

Source

Triage notes (auto)

  • paperType: benchmark
  • estimatedCaseCount: 3
  • triage reason: Introduces GDB benchmark with 230 reproducible test scenarios measuring goal displacement across 18 models; identifies three distinct failure modes (trap displacement, reverse displacement, mixed empathy-with-constraints) with human-scored annotations and evaluation code.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet