SmolVLM: Redefining small and efficient multimodal models
Large Vision-Language Models (VLMs) deliver exceptional performance but
require significant computational resources, limiting their deployment on
mobile and edge devices. Smaller VLMs typically mirror design choices of larger
models, such as extensive image tokenization, leading to inefficient GPU memory
usage and constrained practicality for on-device applications.
  We introduce SmolVLM, a series of compact multimodal models specifically
engineered for resource-efficient inference. We systematically explore
architectural configurations, tokenization strategies, and data curation
optimized for low computational overhead. Through this, we identify key design
choices that yield substantial performance gains on image and video tasks with
minimal memory footprints.
  Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during
inference and outperforms the 300-times larger Idefics-80B model, despite an
18-month development gap. Our largest model, at 2.2B parameters, rivals
state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend
beyond static images, demonstrating robust video comprehension capabilities.
  Our results emphasize that strategic architectural optimizations, aggressive
yet efficient tokenization, and carefully curated training data significantly
enhance multimodal performance, facilitating practical, energy-efficient
deployments at significantly smaller scales.
67f4cf5b504263bce1236d87
Andrés Marafioti
claimed_verified
2025-04-08T08:46:33.366Z
65d66b494bbd0d92b641cdbb
https://cdn-avatars.huggingface.co/v1/production/uploads/65d66b494bbd0d92b641cdbb/6-7dm7B-JxcoS1QlCPdMN.jpeg
Andres Marafioti
user
andito
67f4cf5b504263bce1236d88
Orr Zohar
claimed_verified
2025-04-08T09:11:26.498Z
648c9605565e3a44f3c9bb7b
https://cdn-avatars.huggingface.co/v1/production/uploads/648c9605565e3a44f3c9bb7b/W5chvk17Zol6-2QSWkFVR.jpeg
Orr Zohar
user
orrzohar
67f4cf5b504263bce1236d89
Miquel Farré
claimed_verified
2025-04-08T09:11:24.824Z
61ed0ff29539bc0a3bbc89f4
https://cdn-avatars.huggingface.co/v1/production/uploads/61ed0ff29539bc0a3bbc89f4/iYWK7GParA7Ke5F6q132W.jpeg
Miquel Farré
user
mfarre
67f4cf5b504263bce1236d8a
Merve Noyan
admin_assigned
2025-04-08T10:29:11.673Z
60315285d2c57896177ce764
https://cdn-avatars.huggingface.co/v1/production/uploads/1613845120202-noauth.png
Merve Noyan
user
mervenoyan
67f4cf5b504263bce1236d8b
Elie Bakouch
admin_assigned
2025-04-08T10:29:19.329Z
651e96991b97c9f33d26bde6
https://cdn-avatars.huggingface.co/v1/production/uploads/651e96991b97c9f33d26bde6/-Bqs6qrmz0yCfwtB2e-6q.jpeg
Elie Bakouch
user
eliebak
67f4cf5b504263bce1236d8c
Pedro Cuenca
admin_assigned
2025-04-08T10:29:25.903Z
603d25b75f9d390ab190b777
https://cdn-avatars.huggingface.co/v1/production/uploads/1617264212503-603d25b75f9d390ab190b777.jpeg
Pedro Cuenca
user
pcuenq
67f4cf5b504263bce1236d8d
Cyril Zakka
admin_assigned
2025-04-08T10:29:33.323Z
66ba71a4447411b9c0e19d71
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4f93ZrYdaKfK3F53IB51x.jpeg
Cyril
user
cyrilzakka
67f4cf5b504263bce1236d8e
Loubna Ben Allal
claimed_verified
2025-04-08T09:11:22.957Z
61c141342aac764ce1654e43
https://cdn-avatars.huggingface.co/v1/production/uploads/61c141342aac764ce1654e43/81AwoT5IQ_Xdw0OVw7TKu.jpeg
Loubna Ben Allal
user
loubnabnl
67f4cf5b504263bce1236d8f
Anton Lozhkov
admin_assigned
2025-04-08T10:29:39.905Z
602e6dee60e3dd96631c906e
https://cdn-avatars.huggingface.co/v1/production/uploads/1613655355830-noauth.png
Anton Lozhkov
user
anton-l
67f4cf5b504263bce1236d90
Nouamane Tazi
admin_assigned
2025-04-08T10:29:46.326Z
5ff8c9f4b2035d9a81a859f7
https://cdn-avatars.huggingface.co/v1/production/uploads/1652134289581-5ff8c9f4b2035d9a81a859f7.jpeg
Nouamane Tazi
user
nouamanetazi
67f4cf5b504263bce1236d91
Vaibhav Srivastav
claimed_verified
2025-04-08T10:28:41.404Z
61b85ce86eb1f2c5e6233736
https://cdn-avatars.huggingface.co/v1/production/uploads/1655385361868-61b85ce86eb1f2c5e6233736.jpeg
Vaibhav Srivastav
user
reach-vb
67f4cf5b504263bce1236d92
Joshua Lochner
claimed_verified
2025-04-08T10:28:45.098Z
61b253b7ac5ecaae3d1efe0c
https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png
Joshua
user
Xenova
67f4cf5b504263bce1236d93
Hugo Larcher
admin_assigned
2025-04-08T10:29:52.171Z
641cc77c92cd25302998b740
https://cdn-avatars.huggingface.co/v1/production/uploads/641cc77c92cd25302998b740/5A81W5s3ecLaLXFir52Rw.jpeg
Hugo Larcher
user
hlarcher
67f4cf5b504263bce1236d94
Mathieu Morlon
admin_assigned
2025-04-08T10:29:57.811Z
664d7d1e4f54c9372970e121
https://cdn-avatars.huggingface.co/v1/production/uploads/664d7d1e4f54c9372970e121/TEWsq1_zSBBeuHLu0HlDc.png
Mathieu Morlon
user
glutamatt
67f4cf5b504263bce1236d95
Lewis Tunstall
admin_assigned
2025-04-08T10:30:03.679Z
5f0c746619cb630495b814fd
https://cdn-avatars.huggingface.co/v1/production/uploads/1594651707950-noauth.jpeg
Lewis Tunstall
user
lewtun
67f4cf5b504263bce1236d96
Leandro von Werra
claimed_verified
2025-04-08T10:28:43.360Z
5e48005437cb5b49818287a5
https://cdn-avatars.huggingface.co/v1/production/uploads/5e48005437cb5b49818287a5/4uCXGGui-9QifAT4qelxU.png
Leandro von Werra
user
lvwerra
67f4cf5b504263bce1236d97
Thomas Wolf
admin_assigned
2025-04-08T10:30:10.389Z
5df7e9e5da6d0311fd3d53f9
https://cdn-avatars.huggingface.co/v1/production/uploads/1583857746553-5df7e9e5da6d0311fd3d53f9.jpeg
Thomas Wolf
user
thomwolf
SmolVLM: Redefining small and efficient multimodal models
Large Vision-Language Models (VLMs) deliver exceptional performance but
require significant computational resources, limiting their deployment on
mobile and edge devices. Smaller VLMs typically mirror design choices of larger
models, such as extensive image tokenization, leading to inefficient GPU memory
usage and constrained practicality for on-device applications.
  We introduce SmolVLM, a series of compact multimodal models specifically
engineered for resource-efficient inference. We systematically explore
architectural configurations, tokenization strategies, and data curation
optimized for low computational overhead. Through this, we identify key design
choices that yield substantial performance gains on image and video tasks with
minimal memory footprints.
  Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during
inference and outperforms the 300-times larger Idefics-80B model, despite an
18-month development gap. Our largest model, at 2.2B parameters, rivals
state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend
beyond static images, demonstrating robust video comprehension capabilities.
  Our results emphasize that strategic architectural optimizations, aggressive
yet efficient tokenization, and carefully curated training data significantly
enhance multimodal performance, facilitating practical, energy-efficient
deployments at significantly smaller scales.
SmolVLM, a series of compact multimodal models, achieves high performance with minimal GPU memory usage, making efficient deployment on mobile and edge devices possible.
Large Vision-Language Models VLMs SmolVLM multimodal models image tokenization GPU memory inference architectural configurations tokenization strategies data curation video comprehension Idefics-80B parameter-efficient inference