title: Tracking Any Object Amodally
summary: Amodal perception, the ability to comprehend complete object structures from
  partial visibility, is a fundamental skill, even for infants. Its significance
  extends to applications like autonomous driving, where a clear understanding of
  heavily occluded objects is essential. However, modern detection and tracking
  algorithms often overlook this critical capability, perhaps due to the
  prevalence of modal annotations in most datasets. To address the scarcity of
  amodal data, we introduce the TAO-Amodal benchmark, featuring 880 diverse
  categories in thousands of video sequences. Our dataset includes amodal and
  modal bounding boxes for visible and occluded objects, including objects that
  are partially out-of-frame. To enhance amodal tracking with object permanence,
  we leverage a lightweight plug-in module, the amodal expander, to transform
  standard, modal trackers into amodal ones through fine-tuning on a few hundred
  video sequences with data augmentation. We achieve a 3.3\% and 1.6\%
  improvement on the detection and tracking of occluded objects on TAO-Amodal.
  When evaluated on people, our method produces dramatic improvements of 2x
  compared to state-of-the-art modal baselines.
paper_authors._id: 6582627192b5a9664de9ffe5
paper_authors.name: Cheng-Yen Hsieh
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:20:42.742Z
paper_authors._id: 649d72f62634f8711f32b89c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/z7OZ0L_yl8UFzWKAWebNW.jpeg
paper_authors.fullname: Cheng-Yen Hsieh
paper_authors.type: user
paper_authors.user: chengyenhsieh
paper_authors._id: 6582627192b5a9664de9ffe6
paper_authors.name: Tarasha Khurana
paper_authors._id: 6582627192b5a9664de9ffe7
paper_authors.name: Achal Dave
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:21:23.333Z
paper_authors._id: 6488e29b78570cedad71eb2d
paper_authors.avatarUrl: /avatars/0d8bb52c1e2be8d6aa7c732cf92fc5a9.svg
paper_authors.fullname: Achal Dave
paper_authors.type: user
paper_authors.user: achal-tri
paper_authors._id: 6582627192b5a9664de9ffe8
paper_authors.name: Deva Ramanan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-20T09:21:29.842Z
paper_authors._id: 6337151b0267ebcf02640eb6
paper_authors.avatarUrl: /avatars/14a723cafc5587043bdfb19304fc202d.svg
paper_authors.fullname: Deva Ramanan
paper_authors.type: user
paper_authors.user: devakramanan
paper_title: Tracking Any Object Amodally
paper_summary: Amodal perception, the ability to comprehend complete object structures from
  partial visibility, is a fundamental skill, even for infants. Its significance
  extends to applications like autonomous driving, where a clear understanding of
  heavily occluded objects is essential. However, modern detection and tracking
  algorithms often overlook this critical capability, perhaps due to the
  prevalence of modal annotations in most datasets. To address the scarcity of
  amodal data, we introduce the TAO-Amodal benchmark, featuring 880 diverse
  categories in thousands of video sequences. Our dataset includes amodal and
  modal bounding boxes for visible and occluded objects, including objects that
  are partially out-of-frame. To enhance amodal tracking with object permanence,
  we leverage a lightweight plug-in module, the amodal expander, to transform
  standard, modal trackers into amodal ones through fine-tuning on a few hundred
  video sequences with data augmentation. We achieve a 3.3\% and 1.6\%
  improvement on the detection and tracking of occluded objects on TAO-Amodal.
  When evaluated on people, our method produces dramatic improvements of 2x
  compared to state-of-the-art modal baselines.
paper_ai_summary: A new TAO-Amodal benchmark and an amodal expander module improve amodal detection and tracking of occluded objects in video sequences.
paper_ai_keywords: amodal perception TAO-Amodal benchmark amodal bounding boxes modal bounding boxes object permanence amodal expander fine-tuning data augmentation