title: SMITE: Segment Me In TimE
summary: Segmenting an object in a video presents significant challenges. Each pixel
  must be accurately labelled, and these labels must remain consistent across
  frames. The difficulty increases when the segmentation is with arbitrary
  granularity, meaning the number of segments can vary arbitrarily, and masks are
  defined based on only one or a few sample images. In this paper, we address
  this issue by employing a pre-trained text to image diffusion model
  supplemented with an additional tracking mechanism. We demonstrate that our
  approach can effectively manage various segmentation scenarios and outperforms
  state-of-the-art alternatives.
paper_authors._id: 671b01ae8750e04558bb91d7
paper_authors.name: Amirhossein Alimohammadi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:29:55.956Z
paper_authors._id: 650e3abcae507a2c7c847baa
paper_authors.avatarUrl: /avatars/a138bc6c9ba80a486fb36f4e1aa16b33.svg
paper_authors.fullname: Amirhossein Alimohammadi
paper_authors.type: user
paper_authors.user: Amirhossein-Alimohammadi
paper_authors._id: 671b01ae8750e04558bb91d8
paper_authors.name: Sauradip Nag
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-25T09:29:53.932Z
paper_authors._id: 6399ab3296ce14c5dcf4ccbf
paper_authors.avatarUrl: /avatars/89aeedc96f73d76f7a6da96454f40fd2.svg
paper_authors.fullname: Sauradip Nag
paper_authors.type: user
paper_authors.user: sauradip
paper_authors._id: 671b01ae8750e04558bb91d9
paper_authors.name: Saeid Asgari Taghanaki
paper_authors._id: 671b01ae8750e04558bb91da
paper_authors.name: Andrea Tagliasacchi
paper_authors._id: 671b01ae8750e04558bb91db
paper_authors.name: Ghassan Hamarneh
paper_authors._id: 671b01ae8750e04558bb91dc
paper_authors.name: Ali Mahdavi Amiri
paper_title: SMITE: Segment Me In TimE
paper_summary: Segmenting an object in a video presents significant challenges. Each pixel
  must be accurately labelled, and these labels must remain consistent across
  frames. The difficulty increases when the segmentation is with arbitrary
  granularity, meaning the number of segments can vary arbitrarily, and masks are
  defined based on only one or a few sample images. In this paper, we address
  this issue by employing a pre-trained text to image diffusion model
  supplemented with an additional tracking mechanism. We demonstrate that our
  approach can effectively manage various segmentation scenarios and outperforms
  state-of-the-art alternatives.
paper_ai_summary: A pre-trained text-to-image diffusion model with tracking mechanism outperforms existing methods in object segmentation across video frames with arbitrary granularity.
paper_ai_keywords: diffusion model tracking mechanism object segmentation video frames arbitrary granularity