title: ReSurgSAM2: Referring Segment Anything in Surgical Video via Credible
    Long-term Tracking
summary: Surgical scene segmentation is critical in computer-assisted surgery and is
  vital for enhancing surgical quality and patient outcomes. Recently, referring
  surgical segmentation is emerging, given its advantage of providing surgeons
  with an interactive experience to segment the target object. However, existing
  methods are limited by low efficiency and short-term tracking, hindering their
  applicability in complex real-world surgical scenarios. In this paper, we
  introduce ReSurgSAM2, a two-stage surgical referring segmentation framework
  that leverages Segment Anything Model 2 to perform text-referred target
  detection, followed by tracking with reliable initial frame identification and
  diversity-driven long-term memory. For the detection stage, we propose a
  cross-modal spatial-temporal Mamba to generate precise detection and
  segmentation results. Based on these results, our credible initial frame
  selection strategy identifies the reliable frame for the subsequent tracking.
  Upon selecting the initial frame, our method transitions to the tracking stage,
  where it incorporates a diversity-driven memory mechanism that maintains a
  credible and diverse memory bank, ensuring consistent long-term tracking.
  Extensive experiments demonstrate that ReSurgSAM2 achieves substantial
  improvements in accuracy and efficiency compared to existing methods, operating
  in real-time at 61.2 FPS. Our code and datasets will be available at
  https://github.com/jinlab-imvr/ReSurgSAM2.
paper_authors._id: 6825c47fbe0e3c3bfbf4d285
paper_authors.name: Haofeng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-16T07:12:36.358Z
paper_authors._id: 66bf75432777c05070bf49dc
paper_authors.avatarUrl: /avatars/e18c2f6f5b1c37873dc181de4f255a8b.svg
paper_authors.fullname: Haofeng Liu
paper_authors.type: user
paper_authors.user: HeverLaw
paper_authors._id: 6825c47fbe0e3c3bfbf4d286
paper_authors.name: Mingqi Gao
paper_authors._id: 6825c47fbe0e3c3bfbf4d287
paper_authors.name: Xuxiao Luo
paper_authors._id: 6825c47fbe0e3c3bfbf4d288
paper_authors.name: Ziyue Wang
paper_authors._id: 6825c47fbe0e3c3bfbf4d289
paper_authors.name: Guanyi Qin
paper_authors._id: 6825c47fbe0e3c3bfbf4d28a
paper_authors.name: Junde Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T14:20:21.901Z
paper_authors._id: 6317257fc92fd6fee317ff7c
paper_authors.avatarUrl: /avatars/2f460a2f28562c987becb2acad8d93e7.svg
paper_authors.fullname: Junde Wu
paper_authors.type: user
paper_authors.user: morson
paper_authors._id: 6825c47fbe0e3c3bfbf4d28b
paper_authors.name: Yueming Jin
paper_title: ReSurgSAM2: Referring Segment Anything in Surgical Video via Credible
    Long-term Tracking
paper_summary: Surgical scene segmentation is critical in computer-assisted surgery and is
  vital for enhancing surgical quality and patient outcomes. Recently, referring
  surgical segmentation is emerging, given its advantage of providing surgeons
  with an interactive experience to segment the target object. However, existing
  methods are limited by low efficiency and short-term tracking, hindering their
  applicability in complex real-world surgical scenarios. In this paper, we
  introduce ReSurgSAM2, a two-stage surgical referring segmentation framework
  that leverages Segment Anything Model 2 to perform text-referred target
  detection, followed by tracking with reliable initial frame identification and
  diversity-driven long-term memory. For the detection stage, we propose a
  cross-modal spatial-temporal Mamba to generate precise detection and
  segmentation results. Based on these results, our credible initial frame
  selection strategy identifies the reliable frame for the subsequent tracking.
  Upon selecting the initial frame, our method transitions to the tracking stage,
  where it incorporates a diversity-driven memory mechanism that maintains a
  credible and diverse memory bank, ensuring consistent long-term tracking.
  Extensive experiments demonstrate that ReSurgSAM2 achieves substantial
  improvements in accuracy and efficiency compared to existing methods, operating
  in real-time at 61.2 FPS. Our code and datasets will be available at
  https://github.com/jinlab-imvr/ReSurgSAM2.
paper_ai_summary: ReSurgSAM2 is a two-stage framework that improves surgical scene segmentation and tracking using Segment Anything Model 2, cross-modal spatial-temporal Mamba, and a diversity-driven memory mechanism.
paper_ai_keywords: ReSurgSAM2 Segment Anything Model 2 cross-modal spatial-temporal Mamba diversity-driven long-term memory