title: TinySAM: Pushing the Envelope for Efficient Segment Anything Model
summary: Recently segment anything model (SAM) has shown powerful segmentation
  capability and has drawn great attention in computer vision fields. Massive
  following works have developed various applications based on the pretrained SAM
  and achieved impressive performance on downstream vision tasks. However, SAM
  consists of heavy architectures and requires massive computational capacity,
  which hinders the further application of SAM on computation constrained edge
  devices. To this end, in this paper we propose a framework to obtain a tiny
  segment anything model (TinySAM) while maintaining the strong zero-shot
  performance. We first propose a full-stage knowledge distillation method with
  online hard prompt sampling strategy to distill a lightweight student model. We
  also adapt the post-training quantization to the promptable segmentation task
  and further reduce the computational cost. Moreover, a hierarchical segmenting
  everything strategy is proposed to accelerate the everything inference by
  2times with almost no performance degradation. With all these proposed
  methods, our TinySAM leads to orders of magnitude computational reduction and
  pushes the envelope for efficient segment anything task. Extensive experiments
  on various zero-shot transfer tasks demonstrate the significantly advantageous
  performance of our TinySAM against counterpart methods. Pre-trained models and
  codes will be available at https://github.com/xinghaochen/TinySAM and
  https://gitee.com/mindspore/models/tree/master/research/cv/TinySAM.
paper_authors._id: 6584f07641dbedb146fb2ca0
paper_authors.name: Han Shu
paper_authors._id: 6584f07641dbedb146fb2ca1
paper_authors.name: Wenshuo Li
paper_authors._id: 6584f07641dbedb146fb2ca2
paper_authors.name: Yehui Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:21:14.139Z
paper_authors._id: 64d5deb154bb9eb704f83122
paper_authors.avatarUrl: /avatars/86ce09bcca903319051e2307581a43f4.svg
paper_authors.fullname: Yehui Tang
paper_authors.type: user
paper_authors.user: tangyehui
paper_authors._id: 6584f07641dbedb146fb2ca3
paper_authors.name: Yiman Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:21:20.400Z
paper_authors._id: 64be15c98496ee0fb60e4c22
paper_authors.avatarUrl: /avatars/09e0b747e43c3aadeaa8a715226ee0c0.svg
paper_authors.fullname: Yiman Zhang
paper_authors.type: user
paper_authors.user: YimanZhang
paper_authors._id: 6584f07641dbedb146fb2ca4
paper_authors.name: Yihao Chen
paper_authors._id: 6584f07641dbedb146fb2ca5
paper_authors.name: Houqiang Li
paper_authors._id: 6584f07641dbedb146fb2ca6
paper_authors.name: Yunhe Wang
paper_authors._id: 6584f07641dbedb146fb2ca7
paper_authors.name: Xinghao Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-16T10:46:21.300Z
paper_authors._id: 65878c774f9d2b955e025019
paper_authors.avatarUrl: /avatars/c039c29e131a8cba96b38048da94971b.svg
paper_authors.fullname: Xinghao Chen
paper_authors.type: user
paper_authors.user: xinghaochen
paper_title: TinySAM: Pushing the Envelope for Efficient Segment Anything Model
paper_summary: Recently segment anything model (SAM) has shown powerful segmentation
  capability and has drawn great attention in computer vision fields. Massive
  following works have developed various applications based on the pretrained SAM
  and achieved impressive performance on downstream vision tasks. However, SAM
  consists of heavy architectures and requires massive computational capacity,
  which hinders the further application of SAM on computation constrained edge
  devices. To this end, in this paper we propose a framework to obtain a tiny
  segment anything model (TinySAM) while maintaining the strong zero-shot
  performance. We first propose a full-stage knowledge distillation method with
  online hard prompt sampling strategy to distill a lightweight student model. We
  also adapt the post-training quantization to the promptable segmentation task
  and further reduce the computational cost. Moreover, a hierarchical segmenting
  everything strategy is proposed to accelerate the everything inference by
  2times with almost no performance degradation. With all these proposed
  methods, our TinySAM leads to orders of magnitude computational reduction and
  pushes the envelope for efficient segment anything task. Extensive experiments
  on various zero-shot transfer tasks demonstrate the significantly advantageous
  performance of our TinySAM against counterpart methods. Pre-trained models and
  codes will be available at https://github.com/xinghaochen/TinySAM and
  https://gitee.com/mindspore/models/tree/master/research/cv/TinySAM.
paper_ai_summary: TinySAM reduces computational cost while maintaining strong zero-shot segmentation performance through knowledge distillation, quantization, and hierarchical segmentation strategies.
paper_ai_keywords: knowledge distillation online hard prompt sampling post-training quantization hierarchical segmenting everything strategy zero-shot segmentation