title: SmolVLM: Redefining small and efficient multimodal models
summary: Large Vision-Language Models (VLMs) deliver exceptional performance but
  require significant computational resources, limiting their deployment on
  mobile and edge devices. Smaller VLMs typically mirror design choices of larger
  models, such as extensive image tokenization, leading to inefficient GPU memory
  usage and constrained practicality for on-device applications.
    We introduce SmolVLM, a series of compact multimodal models specifically
  engineered for resource-efficient inference. We systematically explore
  architectural configurations, tokenization strategies, and data curation
  optimized for low computational overhead. Through this, we identify key design
  choices that yield substantial performance gains on image and video tasks with
  minimal memory footprints.
    Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during
  inference and outperforms the 300-times larger Idefics-80B model, despite an
  18-month development gap. Our largest model, at 2.2B parameters, rivals
  state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend
  beyond static images, demonstrating robust video comprehension capabilities.
    Our results emphasize that strategic architectural optimizations, aggressive
  yet efficient tokenization, and carefully curated training data significantly
  enhance multimodal performance, facilitating practical, energy-efficient
  deployments at significantly smaller scales.
paper_authors._id: 67f4cf5b504263bce1236d87
paper_authors.name: Andrés Marafioti
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T08:46:33.366Z
paper_authors._id: 65d66b494bbd0d92b641cdbb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65d66b494bbd0d92b641cdbb/6-7dm7B-JxcoS1QlCPdMN.jpeg
paper_authors.fullname: Andres Marafioti
paper_authors.type: user
paper_authors.user: andito
paper_authors._id: 67f4cf5b504263bce1236d88
paper_authors.name: Orr Zohar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T09:11:26.498Z
paper_authors._id: 648c9605565e3a44f3c9bb7b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648c9605565e3a44f3c9bb7b/W5chvk17Zol6-2QSWkFVR.jpeg
paper_authors.fullname: Orr Zohar
paper_authors.type: user
paper_authors.user: orrzohar
paper_authors._id: 67f4cf5b504263bce1236d89
paper_authors.name: Miquel Farré
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T09:11:24.824Z
paper_authors._id: 61ed0ff29539bc0a3bbc89f4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61ed0ff29539bc0a3bbc89f4/iYWK7GParA7Ke5F6q132W.jpeg
paper_authors.fullname: Miquel Farré
paper_authors.type: user
paper_authors.user: mfarre
paper_authors._id: 67f4cf5b504263bce1236d8a
paper_authors.name: Merve Noyan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:11.673Z
paper_authors._id: 60315285d2c57896177ce764
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1613845120202-noauth.png
paper_authors.fullname: Merve Noyan
paper_authors.type: user
paper_authors.user: mervenoyan
paper_authors._id: 67f4cf5b504263bce1236d8b
paper_authors.name: Elie Bakouch
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:19.329Z
paper_authors._id: 651e96991b97c9f33d26bde6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/651e96991b97c9f33d26bde6/-Bqs6qrmz0yCfwtB2e-6q.jpeg
paper_authors.fullname: Elie Bakouch
paper_authors.type: user
paper_authors.user: eliebak
paper_authors._id: 67f4cf5b504263bce1236d8c
paper_authors.name: Pedro Cuenca
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:25.903Z
paper_authors._id: 603d25b75f9d390ab190b777
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1617264212503-603d25b75f9d390ab190b777.jpeg
paper_authors.fullname: Pedro Cuenca
paper_authors.type: user
paper_authors.user: pcuenq
paper_authors._id: 67f4cf5b504263bce1236d8d
paper_authors.name: Cyril Zakka
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:33.323Z
paper_authors._id: 66ba71a4447411b9c0e19d71
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4f93ZrYdaKfK3F53IB51x.jpeg
paper_authors.fullname: Cyril
paper_authors.type: user
paper_authors.user: cyrilzakka
paper_authors._id: 67f4cf5b504263bce1236d8e
paper_authors.name: Loubna Ben Allal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T09:11:22.957Z
paper_authors._id: 61c141342aac764ce1654e43
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61c141342aac764ce1654e43/81AwoT5IQ_Xdw0OVw7TKu.jpeg
paper_authors.fullname: Loubna Ben Allal
paper_authors.type: user
paper_authors.user: loubnabnl
paper_authors._id: 67f4cf5b504263bce1236d8f
paper_authors.name: Anton Lozhkov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:39.905Z
paper_authors._id: 602e6dee60e3dd96631c906e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1613655355830-noauth.png
paper_authors.fullname: Anton Lozhkov
paper_authors.type: user
paper_authors.user: anton-l
paper_authors._id: 67f4cf5b504263bce1236d90
paper_authors.name: Nouamane Tazi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:46.326Z
paper_authors._id: 5ff8c9f4b2035d9a81a859f7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1652134289581-5ff8c9f4b2035d9a81a859f7.jpeg
paper_authors.fullname: Nouamane Tazi
paper_authors.type: user
paper_authors.user: nouamanetazi
paper_authors._id: 67f4cf5b504263bce1236d91
paper_authors.name: Vaibhav Srivastav
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T10:28:41.404Z
paper_authors._id: 61b85ce86eb1f2c5e6233736
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1655385361868-61b85ce86eb1f2c5e6233736.jpeg
paper_authors.fullname: Vaibhav Srivastav
paper_authors.type: user
paper_authors.user: reach-vb
paper_authors._id: 67f4cf5b504263bce1236d92
paper_authors.name: Joshua Lochner
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T10:28:45.098Z
paper_authors._id: 61b253b7ac5ecaae3d1efe0c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61b253b7ac5ecaae3d1efe0c/hwiQ0uvz3t-L5a-NtBIO6.png
paper_authors.fullname: Joshua
paper_authors.type: user
paper_authors.user: Xenova
paper_authors._id: 67f4cf5b504263bce1236d93
paper_authors.name: Hugo Larcher
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:52.171Z
paper_authors._id: 641cc77c92cd25302998b740
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641cc77c92cd25302998b740/5A81W5s3ecLaLXFir52Rw.jpeg
paper_authors.fullname: Hugo Larcher
paper_authors.type: user
paper_authors.user: hlarcher
paper_authors._id: 67f4cf5b504263bce1236d94
paper_authors.name: Mathieu Morlon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:29:57.811Z
paper_authors._id: 664d7d1e4f54c9372970e121
paper_authors.avatarUrl: /avatars/695a209d6951a4623eceedcd2eed3a68.svg
paper_authors.fullname: Mathieu Morlon
paper_authors.type: user
paper_authors.user: glutamatt
paper_authors._id: 67f4cf5b504263bce1236d95
paper_authors.name: Lewis Tunstall
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:30:03.679Z
paper_authors._id: 5f0c746619cb630495b814fd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1594651707950-noauth.jpeg
paper_authors.fullname: Lewis Tunstall
paper_authors.type: user
paper_authors.user: lewtun
paper_authors._id: 67f4cf5b504263bce1236d96
paper_authors.name: Leandro von Werra
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T10:28:43.360Z
paper_authors._id: 5e48005437cb5b49818287a5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/5e48005437cb5b49818287a5/4uCXGGui-9QifAT4qelxU.png
paper_authors.fullname: Leandro von Werra
paper_authors.type: user
paper_authors.user: lvwerra
paper_authors._id: 67f4cf5b504263bce1236d97
paper_authors.name: Thomas Wolf
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-08T10:30:10.389Z
paper_authors._id: 5df7e9e5da6d0311fd3d53f9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1583857746553-5df7e9e5da6d0311fd3d53f9.jpeg
paper_authors.fullname: Thomas Wolf
paper_authors.type: user
paper_authors.user: thomwolf
paper_title: SmolVLM: Redefining small and efficient multimodal models
paper_summary: Large Vision-Language Models (VLMs) deliver exceptional performance but
  require significant computational resources, limiting their deployment on
  mobile and edge devices. Smaller VLMs typically mirror design choices of larger
  models, such as extensive image tokenization, leading to inefficient GPU memory
  usage and constrained practicality for on-device applications.
    We introduce SmolVLM, a series of compact multimodal models specifically
  engineered for resource-efficient inference. We systematically explore
  architectural configurations, tokenization strategies, and data curation
  optimized for low computational overhead. Through this, we identify key design
  choices that yield substantial performance gains on image and video tasks with
  minimal memory footprints.
    Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during
  inference and outperforms the 300-times larger Idefics-80B model, despite an
  18-month development gap. Our largest model, at 2.2B parameters, rivals
  state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend
  beyond static images, demonstrating robust video comprehension capabilities.
    Our results emphasize that strategic architectural optimizations, aggressive
  yet efficient tokenization, and carefully curated training data significantly
  enhance multimodal performance, facilitating practical, energy-efficient
  deployments at significantly smaller scales.
paper_ai_summary: SmolVLM, a series of compact multimodal models, achieves high performance with minimal GPU memory usage, making efficient deployment on mobile and edge devices possible.
paper_ai_keywords: Large Vision-Language Models VLMs SmolVLM multimodal models image tokenization GPU memory inference architectural configurations tokenization strategies data curation video comprehension Idefics-80B parameter-efficient inference