MedMobile: A mobile-sized language model with expert-level clinical
  capabilities
Language models (LMs) have demonstrated expert-level reasoning and recall
abilities in medicine. However, computational costs and privacy concerns are
mounting barriers to wide-scale implementation. We introduce a parsimonious
adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of
running on a mobile device, for medical applications. We demonstrate that
MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for
physicians (~60%), and approaching the scores of models 100 times its size. We
subsequently perform a careful set of ablations, and demonstrate that chain of
thought, ensembling, and fine-tuning lead to the greatest performance gains,
while unexpectedly retrieval augmented generation fails to demonstrate
significant improvements
6711ce7841ad206aabb63260
Krithik Vishwanath
extracted_pending
2024-10-18T02:56:57.499Z
66a02bb71bb371390c4ae519
/avatars/2c4791516aae0b20a9c9ce0542dc966e.svg
Krithik Vishwanath
user
KrithikV
6711ce7841ad206aabb63261
Jaden Stryker
admin_assigned
2024-10-18T12:57:32.723Z
6490f001f049c475bdb5178f
/avatars/97523fda9a37a16282e427e8862b0d84.svg
Jaden Stryker
user
JadenStryker
6711ce7841ad206aabb63262
Anton Alaykin
6711ce7841ad206aabb63263
Daniel Alexander Alber
6711ce7841ad206aabb63264
Eric Karl Oermann
MedMobile: A mobile-sized language model with expert-level clinical
  capabilities
Language models (LMs) have demonstrated expert-level reasoning and recall
abilities in medicine. However, computational costs and privacy concerns are
mounting barriers to wide-scale implementation. We introduce a parsimonious
adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of
running on a mobile device, for medical applications. We demonstrate that
MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for
physicians (~60%), and approaching the scores of models 100 times its size. We
subsequently perform a careful set of ablations, and demonstrate that chain of
thought, ensembling, and fine-tuning lead to the greatest performance gains,
while unexpectedly retrieval augmented generation fails to demonstrate
significant improvements
MedMobile, a 3.8 billion parameter LM optimized for mobile devices, outperforms larger models on MedQA with enhancements from chain of thought, ensembling, and fine-tuning.
language models phi-3-mini MedMobile MedQA USMLE chain of thought ensembling fine-tuning retrieval augmented generation