title: EfficientViT-SAM: Accelerated Segment Anything Model Without Performance
    Loss
summary: We present EfficientViT-SAM, a new family of accelerated segment anything
  models. We retain SAM's lightweight prompt encoder and mask decoder while
  replacing the heavy image encoder with EfficientViT. For the training, we begin
  with the knowledge distillation from the SAM-ViT-H image encoder to
  EfficientViT. Subsequently, we conduct end-to-end training on the SA-1B
  dataset. Benefiting from EfficientViT's efficiency and capacity,
  EfficientViT-SAM delivers 48.9x measured TensorRT speedup on A100 GPU over
  SAM-ViT-H without sacrificing performance. Our code and pre-trained models are
  released at https://github.com/mit-han-lab/efficientvit.
paper_authors._id: 65c46c7bb25522fdb02754b6
paper_authors.name: Zhuoyang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-08T11:49:27.882Z
paper_authors._id: 650e6ab08f3228d807707735
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/650e6ab08f3228d807707735/yFo6aLuyGH06t9yG8AOp7.png
paper_authors.fullname: Zhuoyang Zhang
paper_authors.type: user
paper_authors.user: zhuoyang20
paper_authors._id: 65c46c7bb25522fdb02754b7
paper_authors.name: Han Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-08T11:50:01.472Z
paper_authors._id: 650e2b14c945dfc9386a7e28
paper_authors.avatarUrl: /avatars/0a9be20aa53f4c52a2d1a8b02d4093ea.svg
paper_authors.fullname: Han Cai
paper_authors.type: user
paper_authors.user: han-cai
paper_authors._id: 65c46c7bb25522fdb02754b8
paper_authors.name: Song Han
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-08T11:50:30.700Z
paper_authors._id: 646320386c27a7e33b23de65
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NAxll5YVxULmr3G38UzbT.jpeg
paper_authors.fullname: Song Han
paper_authors.type: user
paper_authors.user: SongHanNo1
paper_title: EfficientViT-SAM: Accelerated Segment Anything Model Without Performance
    Loss
paper_summary: We present EfficientViT-SAM, a new family of accelerated segment anything
  models. We retain SAM's lightweight prompt encoder and mask decoder while
  replacing the heavy image encoder with EfficientViT. For the training, we begin
  with the knowledge distillation from the SAM-ViT-H image encoder to
  EfficientViT. Subsequently, we conduct end-to-end training on the SA-1B
  dataset. Benefiting from EfficientViT's efficiency and capacity,
  EfficientViT-SAM delivers 48.9x measured TensorRT speedup on A100 GPU over
  SAM-ViT-H without sacrificing performance. Our code and pre-trained models are
  released at https://github.com/mit-han-lab/efficientvit.
paper_ai_keywords: EfficientViT-SAM prompt encoder mask decoder image encoder knowledge distillation EfficientViT SA-1B dataset TensorRT A100 GPU pre-trained models