BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large
  Language Models on Mobile Devices
The emergence and growing popularity of multimodal large language models
(MLLMs) have significant potential to enhance various aspects of daily life,
from improving communication to facilitating learning and problem-solving.
Mobile phones, as essential daily companions, represent the most effective and
accessible deployment platform for MLLMs, enabling seamless integration into
everyday tasks. However, deploying MLLMs on mobile phones presents challenges
due to limitations in memory size and computational capability, making it
difficult to achieve smooth and real-time processing without extensive
optimization. In this paper, we present BlueLM-V-3B, an algorithm and system
co-design approach specifically tailored for the efficient deployment of MLLMs
on mobile platforms. To be specific, we redesign the dynamic resolution scheme
adopted by mainstream MLLMs and implement system optimization for
hardware-aware deployment to optimize model inference on mobile phones.
BlueLM-V-3B boasts the following key highlights: (1) Small Size: BlueLM-V-3B
features a language model with 2.7B parameters and a vision encoder with 400M
parameters. (2) Fast Speed: BlueLM-V-3B achieves a generation speed of 24.4
token/s on the MediaTek Dimensity 9300 processor with 4-bit LLM weight
quantization. (3) Strong Performance: BlueLM-V-3B has attained the highest
average score of 66.1 on the OpenCompass benchmark among models with leq 4B
parameters and surpassed a series of models with much larger parameter sizes
(e.g., MiniCPM-V-2.6, InternVL2-8B).
673c0e397c1c0ded76fb21d1
Xudong Lu
claimed_verified
2024-11-19T08:38:19.862Z
642e686bbe01b88c9446db8b
https://cdn-avatars.huggingface.co/v1/production/uploads/642e686bbe01b88c9446db8b/tb1DKe5xt50ykOeXiUuTE.jpeg
Lu Xudong
user
lucky-lance
673c0e397c1c0ded76fb21d2
Yinghao Chen
673c0e397c1c0ded76fb21d3
Cheng Chen
673c0e397c1c0ded76fb21d4
Hui Tan
673c0e397c1c0ded76fb21d5
Boheng Chen
673c0e397c1c0ded76fb21d6
Yina Xie
admin_assigned
2024-11-19T08:54:54.265Z
66faa34b7f5d72452e4ddaf6
/avatars/e5687132d1ebc9b1e5225996899b117a.svg
yina x
user
yina0
673c0e397c1c0ded76fb21d7
Rui Hu
673c0e397c1c0ded76fb21d8
Guanxin Tan
673c0e397c1c0ded76fb21d9
Renshou Wu
673c0e397c1c0ded76fb21da
Yan Hu
673c0e397c1c0ded76fb21db
Yi Zeng
673c0e397c1c0ded76fb21dc
Lei Wu
673c0e397c1c0ded76fb21dd
Liuyang Bian
admin_assigned
2024-11-19T08:53:25.632Z
66d459ca1dbd780574b42e2d
/avatars/d573a92487628bf6c727b77eedf80b9d.svg
bian liuyang
user
liuyangbian
673c0e397c1c0ded76fb21de
Zhaoxiong Wang
673c0e397c1c0ded76fb21df
Long Liu
673c0e397c1c0ded76fb21e0
Yanzhou Yang
673c0e397c1c0ded76fb21e1
Han Xiao
claimed_verified
2025-05-28T09:02:24.251Z
666aa99cd1652853e4f9a8b9
/avatars/7cd5a0c34b5ccb8eff5a353d88d15a93.svg
HanXiao
user
HanXiao1999
673c0e397c1c0ded76fb21e2
Aojun Zhou
claimed_verified
2024-11-19T08:38:17.782Z
637de1520d5bb06fbe5207a9
/avatars/1090851217270c5a858b13e013356d4f.svg
AJ.Zhou
user
AJZhou
673c0e397c1c0ded76fb21e3
Yafei Wen
admin_assigned
2024-11-19T08:53:02.294Z
67134b1ce58b557752c54a1c
/avatars/5cc07748dac424a6f5c7ca09f1c45068.svg
yafei wen
user
wolf1110
673c0e397c1c0ded76fb21e4
Xiaoxin Chen
admin_assigned
2024-11-19T08:52:55.372Z
65389a669c474315d7425f96
/avatars/2fa3828ca489cfe1948129a0eccf264f.svg
chenxiaoxin
user
steelozazala
673c0e397c1c0ded76fb21e5
Shuai Ren
673c0e397c1c0ded76fb21e6
Hongsheng Li
admin_assigned
2024-11-19T08:52:25.727Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large
  Language Models on Mobile Devices
The emergence and growing popularity of multimodal large language models
(MLLMs) have significant potential to enhance various aspects of daily life,
from improving communication to facilitating learning and problem-solving.
Mobile phones, as essential daily companions, represent the most effective and
accessible deployment platform for MLLMs, enabling seamless integration into
everyday tasks. However, deploying MLLMs on mobile phones presents challenges
due to limitations in memory size and computational capability, making it
difficult to achieve smooth and real-time processing without extensive
optimization. In this paper, we present BlueLM-V-3B, an algorithm and system
co-design approach specifically tailored for the efficient deployment of MLLMs
on mobile platforms. To be specific, we redesign the dynamic resolution scheme
adopted by mainstream MLLMs and implement system optimization for
hardware-aware deployment to optimize model inference on mobile phones.
BlueLM-V-3B boasts the following key highlights: (1) Small Size: BlueLM-V-3B
features a language model with 2.7B parameters and a vision encoder with 400M
parameters. (2) Fast Speed: BlueLM-V-3B achieves a generation speed of 24.4
token/s on the MediaTek Dimensity 9300 processor with 4-bit LLM weight
quantization. (3) Strong Performance: BlueLM-V-3B has attained the highest
average score of 66.1 on the OpenCompass benchmark among models with leq 4B
parameters and surpassed a series of models with much larger parameter sizes
(e.g., MiniCPM-V-2.6, InternVL2-8B).
BlueLM-V-3B optimizes the deployment of multimodal large language models on mobile platforms through redesigned dynamic resolution and hardware-aware system optimizations, achieving high performance with a small model size and fast generation speed.
multimodal large language models dynamic resolution hardware-aware deployment OpenCompass benchmark parameter-efficient fine-tuning token/s 4-bit LLM weight quantization