title: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic
    Understanding, Localization, and Dense Features
summary: We introduce SigLIP 2, a family of new multilingual vision-language encoders
  that build on the success of the original SigLIP. In this second iteration, we
  extend the original image-text training objective with several prior,
  independently developed techniques into a unified recipe -- this includes
  captioning-based pretraining, self-supervised losses (self-distillation, masked
  prediction) and online data curation. With these changes, SigLIP 2 models
  outperform their SigLIP counterparts at all model scales in core capabilities,
  including zero-shot classification, image-text retrieval, and transfer
  performance when extracting visual representations for Vision-Language Models
  (VLMs). Furthermore, the new training recipe leads to significant improvements
  on localization and dense prediction tasks. We also train variants which
  support multiple resolutions and preserve the input's native aspect ratio.
  Finally, we train on a more diverse data-mixture that includes de-biasing
  techniques, leading to much better multilingual understanding and improved
  fairness. To allow users to trade off inference cost with performance, we
  release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M),
  and g (1B).
paper_authors._id: 67b7ed0d58f6b70b18dda7b4
paper_authors.name: Michael Tschannen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T10:23:44.125Z
paper_authors._id: 6489893e1ec8356ba5bb9777
paper_authors.avatarUrl: /avatars/54354c1e5774cadd1d83d42054e9d96b.svg
paper_authors.fullname: Michael Tschannen
paper_authors.type: user
paper_authors.user: mitsch
paper_authors._id: 67b7ed0d58f6b70b18dda7b5
paper_authors.name: Alexey Gritsenko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T10:23:50.440Z
paper_authors._id: 62d8f9887b8dc0ba17271415
paper_authors.avatarUrl: /avatars/12ec78d34fd849bad44217b212f31e98.svg
paper_authors.fullname: Alexey Gritsenko
paper_authors.type: user
paper_authors.user: AlexeyG
paper_authors._id: 67b7ed0d58f6b70b18dda7b6
paper_authors.name: Xiao Wang
paper_authors._id: 67b7ed0d58f6b70b18dda7b7
paper_authors.name: Muhammad Ferjad Naeem
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-26T15:37:42.802Z
paper_authors._id: 67bf33a512368ec2fad4fe29
paper_authors.avatarUrl: /avatars/ea5c03744ec1c2bcc0e6c13efc8f7ddc.svg
paper_authors.fullname: Muhammad Ferjad Naeem
paper_authors.type: user
paper_authors.user: ferjad
paper_authors._id: 67b7ed0d58f6b70b18dda7b8
paper_authors.name: Ibrahim Alabdulmohsin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T10:24:03.702Z
paper_authors._id: 630545da20668afe24860235
paper_authors.avatarUrl: /avatars/5d82be2e7412bff1af15cc5eafa60b7d.svg
paper_authors.fullname: Ibrahim Alabdulmohsin
paper_authors.type: user
paper_authors.user: ibomohsin
paper_authors._id: 67b7ed0d58f6b70b18dda7b9
paper_authors.name: Nikhil Parthasarathy
paper_authors._id: 67b7ed0d58f6b70b18dda7ba
paper_authors.name: Talfan Evans
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T10:24:14.273Z
paper_authors._id: 66be3074dac7d71326f613cf
paper_authors.avatarUrl: /avatars/012a531aad3eb1a2751bb3c31a619bf5.svg
paper_authors.fullname: Talfan Evans
paper_authors.type: user
paper_authors.user: talfanevans
paper_authors._id: 67b7ed0d58f6b70b18dda7bb
paper_authors.name: Lucas Beyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:44:51.778Z
paper_authors._id: 642d334ff65714b4585f2de4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/642d334ff65714b4585f2de4/gxBynq5KyoUP0VlAQD3-w.jpeg
paper_authors.fullname: Lucas Beyer
paper_authors.type: user
paper_authors.user: giffmana
paper_authors._id: 67b7ed0d58f6b70b18dda7bc
paper_authors.name: Ye Xia
paper_authors._id: 67b7ed0d58f6b70b18dda7bd
paper_authors.name: Basil Mustafa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:44:44.933Z
paper_authors._id: 63cfcad6e23b90128c66685c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674562206311-noauth.png
paper_authors.fullname: Basil Mustafa
paper_authors.type: user
paper_authors.user: BasilMustafa
paper_authors._id: 67b7ed0d58f6b70b18dda7be
paper_authors.name: Olivier Hénaff
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:44:38.156Z
paper_authors._id: 64f30d8ceb5f2982081db604
paper_authors.avatarUrl: /avatars/eedf65a104d099d8a60bbffe69bc2571.svg
paper_authors.fullname: Olivier Henaff
paper_authors.type: user
paper_authors.user: olivierhenaff
paper_authors._id: 67b7ed0d58f6b70b18dda7bf
paper_authors.name: Jeremiah Harmsen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:44:32.302Z
paper_authors._id: 65d77c0e1e7686c460255fda
paper_authors.avatarUrl: /avatars/1d26a7a7ffdc5ca9e67b97030f21b098.svg
paper_authors.fullname: Jeremiah Harmsen
paper_authors.type: user
paper_authors.user: jharmsen
paper_authors._id: 67b7ed0d58f6b70b18dda7c0
paper_authors.name: Andreas Steiner
paper_authors._id: 67b7ed0d58f6b70b18dda7c1
paper_authors.name: Xiaohua Zhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-21T14:44:07.595Z
paper_authors._id: 65dcd90082bddd501f68174b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/M2bc9PyKeFs1cCXjTfGGq.jpeg
paper_authors.fullname: Xiaohua Zhai
paper_authors.type: user
paper_authors.user: xiaohuazhai
paper_title: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic
    Understanding, Localization, and Dense Features
paper_summary: We introduce SigLIP 2, a family of new multilingual vision-language encoders
  that build on the success of the original SigLIP. In this second iteration, we
  extend the original image-text training objective with several prior,
  independently developed techniques into a unified recipe -- this includes
  captioning-based pretraining, self-supervised losses (self-distillation, masked
  prediction) and online data curation. With these changes, SigLIP 2 models
  outperform their SigLIP counterparts at all model scales in core capabilities,
  including zero-shot classification, image-text retrieval, and transfer
  performance when extracting visual representations for Vision-Language Models
  (VLMs). Furthermore, the new training recipe leads to significant improvements
  on localization and dense prediction tasks. We also train variants which
  support multiple resolutions and preserve the input's native aspect ratio.
  Finally, we train on a more diverse data-mixture that includes de-biasing
  techniques, leading to much better multilingual understanding and improved
  fairness. To allow users to trade off inference cost with performance, we
  release model checkpoints at four sizes: ViT-B (86M), L (303M), So400m (400M),
  and g (1B).
paper_ai_keywords: multilingual vision-language encoders captioning-based pretraining self-supervised losses self-distillation masked prediction online data curation zero-shot classification image-text retrieval transfer performance visual representations Vision-Language Models (VLMs) localization dense prediction tasks de-biasing techniques model checkpoints ViT-B ViT-L So400m ViT-g