title: Surgical SAM 2: Real-time Segment Anything in Surgical Video by
    Efficient Frame Pruning
summary: Surgical video segmentation is a critical task in computer-assisted surgery
  and is vital for enhancing surgical quality and patient outcomes. Recently, the
  Segment Anything Model 2 (SAM2) framework has shown superior advancements in
  image and video segmentation. However, SAM2 struggles with efficiency due to
  the high computational demands of processing high-resolution images and complex
  and long-range temporal dynamics in surgical videos. To address these
  challenges, we introduce Surgical SAM 2 (SurgSAM-2), an advanced model to
  utilize SAM2 with an Efficient Frame Pruning (EFP) mechanism, to facilitate
  real-time surgical video segmentation. The EFP mechanism dynamically manages
  the memory bank by selectively retaining only the most informative frames,
  reducing memory usage and computational cost while maintaining high
  segmentation accuracy. Our extensive experiments demonstrate that SurgSAM-2
  significantly improves both efficiency and segmentation accuracy compared to
  the vanilla SAM2. Remarkably, SurgSAM-2 achieves a 3times FPS compared with
  SAM2, while also delivering state-of-the-art performance after fine-tuning with
  lower-resolution data. These advancements establish SurgSAM-2 as a leading
  model for surgical video analysis, making real-time surgical video segmentation
  in resource-constrained environments a feasible reality.
paper_authors._id: 66bf77162777c05070c00057
paper_authors.name: Haofeng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-16T21:03:00.126Z
paper_authors._id: 66bf75432777c05070bf49dc
paper_authors.avatarUrl: /avatars/e18c2f6f5b1c37873dc181de4f255a8b.svg
paper_authors.fullname: Haofeng Liu
paper_authors.type: user
paper_authors.user: HeverLaw
paper_authors._id: 66bf77162777c05070c00058
paper_authors.name: Erli Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-19T22:13:29.305Z
paper_authors._id: 639fd77cbeb95d698de6671e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639fd77cbeb95d698de6671e/nP7IkgzDa2NhyP5K2CTMi.jpeg
paper_authors.fullname: Zhang Erli
paper_authors.type: user
paper_authors.user: KarlZEL
paper_authors._id: 66bf77162777c05070c00059
paper_authors.name: Junde Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-26T07:15:56.811Z
paper_authors._id: 6317257fc92fd6fee317ff7c
paper_authors.avatarUrl: /avatars/2f460a2f28562c987becb2acad8d93e7.svg
paper_authors.fullname: Junde Wu
paper_authors.type: user
paper_authors.user: morson
paper_authors._id: 66bf77162777c05070c0005a
paper_authors.name: Mingxuan Hong
paper_authors._id: 66bf77162777c05070c0005b
paper_authors.name: Yueming Jin
paper_title: Surgical SAM 2: Real-time Segment Anything in Surgical Video by
    Efficient Frame Pruning
paper_summary: Surgical video segmentation is a critical task in computer-assisted surgery
  and is vital for enhancing surgical quality and patient outcomes. Recently, the
  Segment Anything Model 2 (SAM2) framework has shown superior advancements in
  image and video segmentation. However, SAM2 struggles with efficiency due to
  the high computational demands of processing high-resolution images and complex
  and long-range temporal dynamics in surgical videos. To address these
  challenges, we introduce Surgical SAM 2 (SurgSAM-2), an advanced model to
  utilize SAM2 with an Efficient Frame Pruning (EFP) mechanism, to facilitate
  real-time surgical video segmentation. The EFP mechanism dynamically manages
  the memory bank by selectively retaining only the most informative frames,
  reducing memory usage and computational cost while maintaining high
  segmentation accuracy. Our extensive experiments demonstrate that SurgSAM-2
  significantly improves both efficiency and segmentation accuracy compared to
  the vanilla SAM2. Remarkably, SurgSAM-2 achieves a 3times FPS compared with
  SAM2, while also delivering state-of-the-art performance after fine-tuning with
  lower-resolution data. These advancements establish SurgSAM-2 as a leading
  model for surgical video analysis, making real-time surgical video segmentation
  in resource-constrained environments a feasible reality.
paper_ai_keywords: Segment Anything Model 2 (SAM2) surgical video segmentation image and video segmentation Efficient Frame Pruning (EFP) memory bank segmentation accuracy FPS fine-tuning