SYS:ONLINELAT:n/aBUILD:78a79a3
[CASE-279]·STATUS:ACTIVE·OPENED:2026-10-20·UPDATED:2026-10-20

Continued-pretraining medical fine-tuning amplifies overconfident assertions in patient-facing LLMs

submitted_by:@mexiQQ
alignmentfrom-arxivauto-published
cat case_body.md

Auto-published from arXiv:2610.24807 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.80, flags: [no-prompt-excerpt])

Category

alignment

Model

BioMistral-7B, OpenBioLLM-Llama3-8B

Surface

API

Setup

200 real patient questions from the K-QA benchmark are posed to six 7–9B instruction-tuned models (three general, three medically adapted). All models run with 4-bit NF4 quantization, greedy decoding, seed 42, max_tokens=300. Risk-bearing language is scored by a three-judge LLM ensemble (Gemini 2.5 Flash, GPT-4o-mini, Claude Haiku 3.5) at temperature 0 using a shared rubric. No single verbatim patient prompt is given for this aggregate finding.

Reproduction anchor

benchmark: K-QA (200 real patient questions); models publicly available as BioMistral-7B and OpenBioLLM-Llama3-8B; harness: 4-bit NF4 quantization, greedy decoding, seed 42, max_tokens=300; rubric and judge prompts stated to be released with the paper but no repository URL found in the paper text

Observed behavior

BioMistral-7B shows a statistically significant increase in overconfident_assertion scores vs. its Mistral-7B base (Δ = +0.195, p = 0.008) while simultaneously suppressing high_alert_medication mentions (Δ = −0.154, p < 10⁻⁴). OpenBioLLM-Llama3-8B amplifies overconfident_assertion even more strongly vs. Llama-3.1-8B base (Δ = +0.255, p < 10⁻⁴) while suppressing dosage_expression (Δ = −0.169, p = 0.0003). Aggregate risk (mean RSHS) does not decrease for any medically fine-tuned model; it is reshaped across categories.

Expected behavior

Medical fine-tuning should reduce or not increase the rate of definitive, unhedged medical claims. Suppressing high-alert medication mentions without amplifying overconfidence elsewhere is the minimum expected safety profile.

Reproducibility

medium

Threat model

Production patient-facing chatbots using medically adapted continued-pretraining models (a common deployment pattern) will present a higher rate of definitive, unhedged clinical assertions than their general-purpose base models, misleading patients into acting on confident-sounding advice without seeking professional consultation. The risk is invisible to evaluations focused solely on factual accuracy.

Novelty

First systematic quantification showing that continued-pretraining medical adaptation reshapes rather than reduces aggregate patient-facing risk, with overconfidence amplification as the dominant failure mode — orthogonal to and undetectable by standard factual hallucination metrics.

Source

Triage notes (auto)

  • paperType: benchmark
  • estimatedCaseCount: 2
  • triage reason: Benchmark evaluating risk-bearing language categories across medical LLMs on 200 patient questions, demonstrating that medical fine-tuning reshapes rather than reduces harm risk. Statistical findings with reproducible framework, but limited specific failure case examples suitable for archive entry.
tail -f comments.log

0 comments

─────────────────────────────────────────────────────────────────────

// no comments yet