Medical World Model: Generative Simulation of Tumor Evolution for
  Treatment Planning
Providing effective treatment and making informed clinical decisions are
essential goals of modern medicine and clinical care. We are interested in
simulating disease dynamics for clinical decision-making, leveraging recent
advances in large generative models. To this end, we introduce the Medical
World Model (MeWM), the first world model in medicine that visually predicts
future disease states based on clinical decisions. MeWM comprises (i)
vision-language models to serve as policy models, and (ii) tumor generative
models as dynamics models. The policy model generates action plans, such as
clinical treatments, while the dynamics model simulates tumor progression or
regression under given treatment conditions. Building on this, we propose the
inverse dynamics model that applies survival analysis to the simulated
post-treatment tumor, enabling the evaluation of treatment efficacy and the
selection of the optimal clinical action plan. As a result, the proposed MeWM
simulates disease dynamics by synthesizing post-treatment tumors, with
state-of-the-art specificity in Turing tests evaluated by radiologists.
Simultaneously, its inverse dynamics model outperforms medical-specialized GPTs
in optimizing individualized treatment protocols across all metrics. Notably,
MeWM improves clinical decision-making for interventional physicians, boosting
F1-score in selecting the optimal TACE protocol by 13%, paving the way for
future integration of medical world models as the second readers.
68476fdd3ec10bdd8ab4dd64
Yijun Yang
68476fdd3ec10bdd8ab4dd65
Zhao-Yang Wang
68476fdd3ec10bdd8ab4dd66
Qiuping Liu
68476fdd3ec10bdd8ab4dd67
Shuwen Sun
68476fdd3ec10bdd8ab4dd68
Kang Wang
68476fdd3ec10bdd8ab4dd69
Rama Chellappa
68476fdd3ec10bdd8ab4dd6a
Zongwei Zhou
68476fdd3ec10bdd8ab4dd6b
Alan Yuille
68476fdd3ec10bdd8ab4dd6c
Lei Zhu
68476fdd3ec10bdd8ab4dd6d
Yu-Dong Zhang
68476fdd3ec10bdd8ab4dd6e
Jieneng Chen
Medical World Model: Generative Simulation of Tumor Evolution for
  Treatment Planning
Providing effective treatment and making informed clinical decisions are
essential goals of modern medicine and clinical care. We are interested in
simulating disease dynamics for clinical decision-making, leveraging recent
advances in large generative models. To this end, we introduce the Medical
World Model (MeWM), the first world model in medicine that visually predicts
future disease states based on clinical decisions. MeWM comprises (i)
vision-language models to serve as policy models, and (ii) tumor generative
models as dynamics models. The policy model generates action plans, such as
clinical treatments, while the dynamics model simulates tumor progression or
regression under given treatment conditions. Building on this, we propose the
inverse dynamics model that applies survival analysis to the simulated
post-treatment tumor, enabling the evaluation of treatment efficacy and the
selection of the optimal clinical action plan. As a result, the proposed MeWM
simulates disease dynamics by synthesizing post-treatment tumors, with
state-of-the-art specificity in Turing tests evaluated by radiologists.
Simultaneously, its inverse dynamics model outperforms medical-specialized GPTs
in optimizing individualized treatment protocols across all metrics. Notably,
MeWM improves clinical decision-making for interventional physicians, boosting
F1-score in selecting the optimal TACE protocol by 13%, paving the way for
future integration of medical world models as the second readers.
MeWM, a medical world model incorporating vision-language and tumor generative models, simulates disease dynamics and optimizes clinical decision-making with state-of-the-art specificity and efficacy.
world model vision-language models policy models tumor generative models dynamics models inverse dynamics model survival analysis Turing tests medical-specialized GPTs F1-score TACE protocol