A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
arXiv 人工智能预印本(Atom) · 2026-10-02 00:07 发布 · 2026-10-05 09:19 采集 · James T. Teo
基本信息
原始数据层(AI 与用户都不会覆盖这些字段)
🌍 国际
技术进展
AI教育
状态:已精选
综合价值 3.6 / 5
- 来源
- arXiv 人工智能预印本(Atom)(arXiv · INTL)
- 发布时间
- 2026-10-02 00:07
- 采集时间
- 2026-10-05 09:19
- 语言
- en
- 地域置信度
- 65%
- 原始链接
- https://arxiv.org/abs/2610.01938v1
- 内容指纹
- 6875c64bffac73ed9c5ab2c264dc8f6c6869e2a0a83aee4a8128242a137befd9
AI 整理
分析于 2026-10-05 09:21(第 1 版,来源:offline)Exam-style accuracy does not establish whether large language models (LLMs) reason well over clinical records.We define clinical reasoning as integrating and updating evidence across time and sources…
为什么值得关注:属于技术进展,可能扩散到其他领域
趋势标签: reasoning clinical representation problem
我的价值判断与记录
系统判断的重要性与你的判断同时保存(§27),永不互相覆盖添加记录
还没有记录。阅读后写下你的评论、启发与后续行动 —— 这是系统最终真正积累的部分。
用户纠错
原始信息 + 用户意见 + 修正结果分别记录,便于审计;系统不会自动改原始数据原始内容
仅在合法且采集到正文时显示;否则请查看原文链接Exam-style accuracy does not establish whether large language models (LLMs) reason well over clinical records. We define clinical reasoning as integrating and updating evidence across time and sources to form, revise and justify a patient's problem representation and a defensible plan.
This structured narrative review maps three literatures: medical education assessment instruments, clinical LLM benchmarks published from 2023 onwards, and general-domain methods for evaluating long-form generation. We examine six dimensions: problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. Preprints are included and flagged.
No single instrument covers all six dimensions. Problem representation and differential or management reasoning are reasonably covered, although reliability varies by instrument and setting. TIMER-Eval targets temporal synthesis, and ER-Reason assesses sequential diagnostic belief updating. Dedicated uncertainty and counterfactual evaluations are emerging, but their applicability to longitudinal free-text reasoning remains limited. Factual completeness is well theorised in general-domain evaluation, with early clinical evidence of important omissions. Faithfulness remains the weakest dimension, with one identified clinical causal-ablation study on multiple-choice questions.
Existing tools should be combined through binary rubric items, separate completeness and correctness scores, case-specific importance weighting with non-compensable safety caps, temporal order-consistency checks, and chance-corrected reliability reporting. Further design work is needed for calibrated uncertainty, counterfactual reasoning and faithfulness over longitudinal free-text records. This review provides a design rationale, not a validated instrument.
事件演化
同一事件的后续报道、状态变化与版本记录
2026-10-02
首次出现
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
暂无后续版本记录。当你把该信息加入「跟踪」后,后续采集到的相关报道会归并到这里。