content.raw: ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 
content.type: text
content.value: ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 
content.raw: https://huggingface.co/collections/ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093
content.id: ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093
content.type: collection
content.type: resource
content.url: https://huggingface.co/collections/ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093
content.type: new_line
content.type: new_line
content.raw: > The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️
content.type: text
content.value: > The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️
content.type: new_line
content.type: new_line
content.raw: > The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)
content.type: text
content.value: > The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)
content.type: new_line
content.type: new_line
content.raw: > The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬
content.type: text
content.value: > The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬
content.type: new_line
content.type: new_line
content.raw: the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️
content.type: text
content.value: the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️
content.type: new_line
content.type: new_line
content.raw: > Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
content.type: text
content.value: > Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
rawContent: ByteDance just dropped SA2VA: a new family of vision LMs combining Qwen2VL/InternVL and SAM2 with MIT license 💗 https://huggingface.co/collections/ByteDance/sa2va-model-zoo-677e3084d71b5f108d00e093
  
  > The models are capable of tasks involving vision-language understanding and visual referrals (referring segmentation) both for images and videos ⏯️
  
  > The models come in 1B, 4B and 8B and are based on InternVL2.5 for base architecture and Qwen2, Qwen2.5 and InternLM2 for language model part (depending on the checkpoint)
  
  > The model is very interesting, it has different encoders for different modalities each (visual prompt, text prompt, image and video) then it concatenates these to feed into LLM 💬
  
  the output segmentation tokens are passed to SAM2, to sort of match text (captions or semantic classes) to masks ⤵️
  
  > Their annotation pipeline is also interesting, they seems to use two open large vision LMs to refine the annotations, and have different levels of descriptions to provide consistency.
reactions.reaction: 🔥
reactions.users: John6666 prithivMLmods reonyy LXT rwightman melmass iky1e
reactions.reaction: 👍
reactions.users: LXT Shakif bobrandomnumber
commentators._id: 6758a9850e3fff481964ca6d
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/EolfJfjW25hC4Bt_hCPq8.png
commentators.fullname: Jean Louis
commentators.name: JLouisBiz
commentators.type: user