Continued-pretraining medical fine-tuning amplifies overconfident assertions in patient-facing LLMs
Auto-published from arXiv:2610.24807 by the mine-arxiv pipeline. Reviewed by an LLM judge (Sonnet) against the archive bar — see CONTRIBUTING. Notes: cleared review (confidence 0.80, flags: [no-prompt-excerpt])
Category
alignment
Model
BioMistral-7B, OpenBioLLM-Llama3-8B
Surface
API
Setup
200 real patient questions from the K-QA benchmark are posed to six 7–9B instruction-tuned models (three general, three medically adapted). All models run with 4-bit NF4 quantization, greedy decoding, seed 42, max_tokens=300. Risk-bearing language is scored by a three-judge LLM ensemble (Gemini 2.5 Flash, GPT-4o-mini, Claude Haiku 3.5) at temperature 0 using a shared rubric. No single verbatim patient prompt is given for this aggregate finding.
Reproduction anchor
benchmark: K-QA (200 real patient questions); models publicly available as BioMistral-7B and OpenBioLLM-Llama3-8B; harness: 4-bit NF4 quantization, greedy decoding, seed 42, max_tokens=300; rubric and judge prompts stated to be released with the paper but no repository URL found in the paper text
Observed behavior
BioMistral-7B shows a statistically significant increase in overconfident_assertion scores vs. its Mistral-7B base (Δ = +0.195, p = 0.008) while simultaneously suppressing high_alert_medication mentions (Δ = −0.154, p < 10⁻⁴). OpenBioLLM-Llama3-8B amplifies overconfident_assertion even more strongly vs. Llama-3.1-8B base (Δ = +0.255, p < 10⁻⁴) while suppressing dosage_expression (Δ = −0.169, p = 0.0003). Aggregate risk (mean RSHS) does not decrease for any medically fine-tuned model; it is reshaped across categories.
Expected behavior
Medical fine-tuning should reduce or not increase the rate of definitive, unhedged medical claims. Suppressing high-alert medication mentions without amplifying overconfidence elsewhere is the minimum expected safety profile.
Reproducibility
medium
Threat model
Production patient-facing chatbots using medically adapted continued-pretraining models (a common deployment pattern) will present a higher rate of definitive, unhedged clinical assertions than their general-purpose base models, misleading patients into acting on confident-sounding advice without seeking professional consultation. The risk is invisible to evaluations focused solely on factual accuracy.
Novelty
First systematic quantification showing that continued-pretraining medical adaptation reshapes rather than reduces aggregate patient-facing risk, with overconfidence amplification as the dominant failure mode — orthogonal to and undetectable by standard factual hallucination metrics.
Source
- arXiv: 2610.24807
- PDF: https://arxiv.org/pdf/2610.24807
- Categories: cs.AI, cs.CL
- Authors: Savan Doshi
Triage notes (auto)
- paperType:
benchmark - estimatedCaseCount: 2
- triage reason: Benchmark evaluating risk-bearing language categories across medical LLMs on 200 patient questions, demonstrating that medical fine-tuning reshapes rather than reduces harm risk. Statistical findings with reproducible framework, but limited specific failure case examples suitable for archive entry.
0 comments
─────────────────────────────────────────────────────────────────────
// no comments yet