EPS Dashboard

Authors: Yicheng Gao, Xiaolin Zhou, Yahan Li et al.

Summary

The authors formalize clinical diagnosis as a Partially Observable Markov Decision Process (POMDP) where the agent can ask questions, order exams (as tool calls), or issue a diagnosis—all under systematic noise (seven patient noise types, three exam noise types). Existing medical LLM benchmarks simplify this to single-turn QA or noise-free conversations, missing the interactive, uncertain nature of real diagnosis. They train MedExAgent via supervised fine-tuning on synthetic Calgary-Cambridge-style clinical interviews, then apply DAPO (a policy optimization method) to maximize a composite reward: diagnostic accuracy, tool-call quality, and exam cost (financial + patient discomfort). MedExAgent matches larger models' diagnostic performance while maintaining cost-efficient exam strategies.

Main takeaways:

Real clinical diagnosis involves questioning, exam ordering, and diagnosis under noisy, incomplete information—existing benchmarks ignore this interactivity.
MedExAgent formalizes diagnosis as a POMDP with three action types and a systematic noise model (e.g., vague patient answers, false-negative test results).
Two-stage training: first supervised fine-tuning on synthetic Calgary-Cambridge interviews, then DAPO to optimize accuracy + tool quality + exam cost.
Achieves diagnostic performance comparable to larger models while keeping exam costs low.
Demonstrates that training on structured interview formats (Calgary-Cambridge) transfers to noisy, interactive diagnosis tasks.

Relevance

Not directly related to my persona installation work—this is about medical diagnosis agents. Could be loosely relevant if I ever think about training assistants to adopt structured interaction patterns (like the Calgary-Cambridge model) as a form of behavioral installation, but that's a stretch.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses benchmark.

Abstract

arXiv:2605.07058v1 Announce Type: new Abstract: Real-world clinical diagnosis is a complex process in which the doctor is required to obtain information from both interaction with the patient and conducting medical exams. Additionally, the doctor needs to adapt to different patient personas, as well as noisy and incomplete information that can happen at any time during the process. However, existing benchmarks for medical LLMs and methods for automatic diagnosis largely simplify this process by reducing it to single-turn question answering, noise-free conversations, or sequential exam making, etc., ignoring the interactive and uncertain nature of clinical diagnosis. In this paper, we aim to address this gap by formalizing clinical diagnosis as a Partially Observable Markov Decision Process (POMDP) with three action types: questioning the patient, ordering medical exams as tool calls, and issuing a diagnosis. We also introduce a systematic noise model comprising seven patient noise types and three exam noise types. Using our proposed environment, we train an effective diagnosis agent, \textbf{MedExAgent}, through a two-stage pipeline that first performs supervised finetuning on synthetic conversations structured after the Calgary-Cambridge model for clinical interviews, and then applies DAPO to optimize a composite reward capturing diagnostic accuracy, tool call quality, and exam cost including financial cost and patient discomfort. Through extensive experiments and ablation studies, we demonstrate that MedExAgent achieves diagnostic performance comparable to larger models while maintaining cost-efficient examination strategies.