MobileVLM : A Fast, Reproducible and Strong Vision Language Assistant
  for Mobile Devices
We present MobileVLM, a competent multimodal vision language model (MMVLM)
targeted to run on mobile devices. It is an amalgamation of a myriad of
architectural designs and techniques that are mobile-oriented, which comprises
a set of language models at the scale of 1.4B and 2.7B parameters, trained from
scratch, a multimodal vision model that is pre-trained in the CLIP fashion,
cross-modality interaction via an efficient projector. We evaluate MobileVLM on
several typical VLM benchmarks. Our models demonstrate on par performance
compared with a few much larger models. More importantly, we measure the
inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin
GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens
per second, respectively. Our code will be made available at:
https://github.com/Meituan-AutoML/MobileVLM.
658e36be04cd2b7fd5fd2b6a
Xiangxiang Chu
658e36be04cd2b7fd5fd2b6b
Limeng Qiao
658e36be04cd2b7fd5fd2b6c
Xinyang Lin
658e36be04cd2b7fd5fd2b6d
Shuang Xu
658e36be04cd2b7fd5fd2b6e
Yang Yang
658e36be04cd2b7fd5fd2b6f
Yiming Hu
658e36be04cd2b7fd5fd2b70
Fei Wei
658e36be04cd2b7fd5fd2b71
Xinyu Zhang
658e36be04cd2b7fd5fd2b72
Bo Zhang
claimed_verified
2024-03-05T08:49:35.308Z
64688d93dcbb937d56b4fbae
https://cdn-avatars.huggingface.co/v1/production/uploads/64688d93dcbb937d56b4fbae/AHf01DkvhL8p_E9ldwggW.jpeg
Bo Zhang
user
boydcheung
658e36be04cd2b7fd5fd2b73
Xiaolin Wei
658e36be04cd2b7fd5fd2b74
Chunhua Shen
MobileVLM : A Fast, Reproducible and Strong Vision Language Assistant
  for Mobile Devices
We present MobileVLM, a competent multimodal vision language model (MMVLM)
targeted to run on mobile devices. It is an amalgamation of a myriad of
architectural designs and techniques that are mobile-oriented, which comprises
a set of language models at the scale of 1.4B and 2.7B parameters, trained from
scratch, a multimodal vision model that is pre-trained in the CLIP fashion,
cross-modality interaction via an efficient projector. We evaluate MobileVLM on
several typical VLM benchmarks. Our models demonstrate on par performance
compared with a few much larger models. More importantly, we measure the
inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin
GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens
per second, respectively. Our code will be made available at:
https://github.com/Meituan-AutoML/MobileVLM.
MobileVLM, a small-scale multimodal vision language model, achieves competitive performance and superior inference speed on mobile devices compared to larger models.
multimodal vision language model MMVLM language models CLIP fashion cross-modality interaction projector