SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-365]·STATUS:ACTIVE·OPENED:2026-11-07·UPDATED:2026-11-07

MCP agents conflate wrong-branch outputs in conditional tasks; Branch Collapse rank is uncorrelated with Unreachable Goal rank (r = -0.12)

submitted_by:@mexiQQ
hallucinationfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2611.07689 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.83, flags: [no-prompt-excerpt])

Category

hallucination

Model

GPT-5-chat (illustrative trace, Appendix B.1); aggregate across 14 frontier LLMs (same set as above)

Surface

agent harness (ReAct-style loop over live MCP servers in Docker containers)

Setup

Agent is given a task with >=2 mutually exclusive branches gated on a live tool output (e.g., rank entities based on a retrieved metric). The correct branch depends on what the tool returns at runtime. The agent must interpret the live observation, select the active branch, and aggregate final results only from that branch. No verbatim prompt in the body; Appendix B.1 contains the full Branch Collapse example with original prompt, structural trigger, execution trace, ground-truth log, and GPT-5 failure.

Reproduction anchor

code: https://github.com/J1zz/mcp_hallu; Appendix B.1 contains the full task example and GPT-5 failure trace; Figure 1(a) illustrates the wrong-ranking failure shape

Observed behavior

Agent follows largely correct intermediate steps but conflates outputs across branch instances at final aggregation, yielding an incorrect ranking. Claude Opus 4.5 scores highest on Branch Collapse (0.867) while overall leader Claude Opus 4.6 (0.728) ranks seventh on this subtype. Branch Collapse and Unreachable Goal are uncorrelated across models (r = -0.12); mean per-model rank shift across subtypes is 6.9 of 13 positions.

Expected behavior

Agent reads the live branch-gating tool output, identifies the active branch, issues tool calls only within that branch's scope, and aggregates results exclusively from active-branch observations.

Reproducibility

medium

Threat model

In production MCP workflows with conditional logic (financial analysis, multi-step search-and-summarize, medical triage), branch conflation causes the agent to report results from the wrong data path as if derived correctly, with no visible error signal to downstream consumers.

Novelty

First benchmark evidence that branch-conditioned reasoning is a failure mode dissociable from infeasibility recognition in MCP agents (r = -0.12), and that the overall top model can rank seventh on this subtype, arguing against single-aggregate MCP evaluation.

Source

Triage notes (auto)

  • paperType: benchmark
  • estimatedCaseCount: 3
  • triage reason: Systematically-designed benchmark of agent hallucination failures across four well-specified subtypes (Branch Collapse, Unreachable Goal, Tool Misuse, Context Forgetting), each with reproducible triggers and quantified failure evidence from 14 frontier models.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet