title: BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large
    Language Models on Mobile Devices
summary: The emergence and growing popularity of multimodal large language models
  (MLLMs) have significant potential to enhance various aspects of daily life,
  from improving communication to facilitating learning and problem-solving.
  Mobile phones, as essential daily companions, represent the most effective and
  accessible deployment platform for MLLMs, enabling seamless integration into
  everyday tasks. However, deploying MLLMs on mobile phones presents challenges
  due to limitations in memory size and computational capability, making it
  difficult to achieve smooth and real-time processing without extensive
  optimization. In this paper, we present BlueLM-V-3B, an algorithm and system
  co-design approach specifically tailored for the efficient deployment of MLLMs
  on mobile platforms. To be specific, we redesign the dynamic resolution scheme
  adopted by mainstream MLLMs and implement system optimization for
  hardware-aware deployment to optimize model inference on mobile phones.
  BlueLM-V-3B boasts the following key highlights: (1) Small Size: BlueLM-V-3B
  features a language model with 2.7B parameters and a vision encoder with 400M
  parameters. (2) Fast Speed: BlueLM-V-3B achieves a generation speed of 24.4
  token/s on the MediaTek Dimensity 9300 processor with 4-bit LLM weight
  quantization. (3) Strong Performance: BlueLM-V-3B has attained the highest
  average score of 66.1 on the OpenCompass benchmark among models with leq 4B
  parameters and surpassed a series of models with much larger parameter sizes
  (e.g., MiniCPM-V-2.6, InternVL2-8B).
paper_authors._id: 673c0e397c1c0ded76fb21d1
paper_authors.name: Xudong Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:38:19.862Z
paper_authors._id: 642e686bbe01b88c9446db8b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642e686bbe01b88c9446db8b/tb1DKe5xt50ykOeXiUuTE.jpeg
paper_authors.fullname: Lu Xudong
paper_authors.type: user
paper_authors.user: lucky-lance
paper_authors._id: 673c0e397c1c0ded76fb21d2
paper_authors.name: Yinghao Chen
paper_authors._id: 673c0e397c1c0ded76fb21d3
paper_authors.name: Cheng Chen
paper_authors._id: 673c0e397c1c0ded76fb21d4
paper_authors.name: Hui Tan
paper_authors._id: 673c0e397c1c0ded76fb21d5
paper_authors.name: Boheng Chen
paper_authors._id: 673c0e397c1c0ded76fb21d6
paper_authors.name: Yina Xie
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T08:54:54.265Z
paper_authors._id: 66faa34b7f5d72452e4ddaf6
paper_authors.avatarUrl: /avatars/e5687132d1ebc9b1e5225996899b117a.svg
paper_authors.fullname: yina x
paper_authors.type: user
paper_authors.user: yina0
paper_authors._id: 673c0e397c1c0ded76fb21d7
paper_authors.name: Rui Hu
paper_authors._id: 673c0e397c1c0ded76fb21d8
paper_authors.name: Guanxin Tan
paper_authors._id: 673c0e397c1c0ded76fb21d9
paper_authors.name: Renshou Wu
paper_authors._id: 673c0e397c1c0ded76fb21da
paper_authors.name: Yan Hu
paper_authors._id: 673c0e397c1c0ded76fb21db
paper_authors.name: Yi Zeng
paper_authors._id: 673c0e397c1c0ded76fb21dc
paper_authors.name: Lei Wu
paper_authors._id: 673c0e397c1c0ded76fb21dd
paper_authors.name: Liuyang Bian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T08:53:25.632Z
paper_authors._id: 66d459ca1dbd780574b42e2d
paper_authors.avatarUrl: /avatars/d573a92487628bf6c727b77eedf80b9d.svg
paper_authors.fullname: bian liuyang
paper_authors.type: user
paper_authors.user: liuyangbian
paper_authors._id: 673c0e397c1c0ded76fb21de
paper_authors.name: Zhaoxiong Wang
paper_authors._id: 673c0e397c1c0ded76fb21df
paper_authors.name: Long Liu
paper_authors._id: 673c0e397c1c0ded76fb21e0
paper_authors.name: Yanzhou Yang
paper_authors._id: 673c0e397c1c0ded76fb21e1
paper_authors.name: Han Xiao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T09:02:24.251Z
paper_authors._id: 666aa99cd1652853e4f9a8b9
paper_authors.avatarUrl: /avatars/7cd5a0c34b5ccb8eff5a353d88d15a93.svg
paper_authors.fullname: HanXiao
paper_authors.type: user
paper_authors.user: HanXiao1999
paper_authors._id: 673c0e397c1c0ded76fb21e2
paper_authors.name: Aojun Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-19T08:38:17.782Z
paper_authors._id: 637de1520d5bb06fbe5207a9
paper_authors.avatarUrl: /avatars/1090851217270c5a858b13e013356d4f.svg
paper_authors.fullname: AJ.Zhou
paper_authors.type: user
paper_authors.user: AJZhou
paper_authors._id: 673c0e397c1c0ded76fb21e3
paper_authors.name: Yafei Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T08:53:02.294Z
paper_authors._id: 67134b1ce58b557752c54a1c
paper_authors.avatarUrl: /avatars/5cc07748dac424a6f5c7ca09f1c45068.svg
paper_authors.fullname: yafei wen
paper_authors.type: user
paper_authors.user: wolf1110
paper_authors._id: 673c0e397c1c0ded76fb21e4
paper_authors.name: Xiaoxin Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T08:52:55.372Z
paper_authors._id: 65389a669c474315d7425f96
paper_authors.avatarUrl: /avatars/2fa3828ca489cfe1948129a0eccf264f.svg
paper_authors.fullname: chenxiaoxin
paper_authors.type: user
paper_authors.user: steelozazala
paper_authors._id: 673c0e397c1c0ded76fb21e5
paper_authors.name: Shuai Ren
paper_authors._id: 673c0e397c1c0ded76fb21e6
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-19T08:52:25.727Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_title: BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large
    Language Models on Mobile Devices
paper_summary: The emergence and growing popularity of multimodal large language models
  (MLLMs) have significant potential to enhance various aspects of daily life,
  from improving communication to facilitating learning and problem-solving.
  Mobile phones, as essential daily companions, represent the most effective and
  accessible deployment platform for MLLMs, enabling seamless integration into
  everyday tasks. However, deploying MLLMs on mobile phones presents challenges
  due to limitations in memory size and computational capability, making it
  difficult to achieve smooth and real-time processing without extensive
  optimization. In this paper, we present BlueLM-V-3B, an algorithm and system
  co-design approach specifically tailored for the efficient deployment of MLLMs
  on mobile platforms. To be specific, we redesign the dynamic resolution scheme
  adopted by mainstream MLLMs and implement system optimization for
  hardware-aware deployment to optimize model inference on mobile phones.
  BlueLM-V-3B boasts the following key highlights: (1) Small Size: BlueLM-V-3B
  features a language model with 2.7B parameters and a vision encoder with 400M
  parameters. (2) Fast Speed: BlueLM-V-3B achieves a generation speed of 24.4
  token/s on the MediaTek Dimensity 9300 processor with 4-bit LLM weight
  quantization. (3) Strong Performance: BlueLM-V-3B has attained the highest
  average score of 66.1 on the OpenCompass benchmark among models with leq 4B
  parameters and surpassed a series of models with much larger parameter sizes
  (e.g., MiniCPM-V-2.6, InternVL2-8B).
paper_ai_summary: BlueLM-V-3B optimizes the deployment of multimodal large language models on mobile platforms through redesigned dynamic resolution and hardware-aware system optimizations, achieving high performance with a small model size and fast generation speed.
paper_ai_keywords: multimodal large language models dynamic resolution hardware-aware deployment OpenCompass benchmark parameter-efficient fine-tuning token/s 4-bit LLM weight quantization