title: STT: Stateful Tracking with Transformers for Autonomous Driving
summary: Tracking objects in three-dimensional space is critical for autonomous
  driving. To ensure safety while driving, the tracker must be able to reliably
  track objects across frames and accurately estimate their states such as
  velocity and acceleration in the present. Existing works frequently focus on
  the association task while either neglecting the model performance on state
  estimation or deploying complex heuristics to predict the states. In this
  paper, we propose STT, a Stateful Tracking model built with Transformers, that
  can consistently track objects in the scenes while also predicting their states
  accurately. STT consumes rich appearance, geometry, and motion signals through
  long term history of detections and is jointly optimized for both data
  association and state estimation tasks. Since the standard tracking metrics
  like MOTA and MOTP do not capture the combined performance of the two tasks in
  the wider spectrum of object states, we extend them with new metrics called
  S-MOTA and MOTPS that address this limitation. STT achieves competitive
  real-time performance on the Waymo Open Dataset.
paper_authors._id: 66332b0295a18522d503f374
paper_authors.name: Longlong Jing
paper_authors._id: 66332b0295a18522d503f375
paper_authors.name: Ruichi Yu
paper_authors._id: 66332b0295a18522d503f376
paper_authors.name: Xu Chen
paper_authors._id: 66332b0295a18522d503f377
paper_authors.name: Zhengli Zhao
paper_authors._id: 66332b0295a18522d503f378
paper_authors.name: Shiwei Sheng
paper_authors._id: 66332b0295a18522d503f379
paper_authors.name: Colin Graber
paper_authors._id: 66332b0295a18522d503f37a
paper_authors.name: Qi Chen
paper_authors._id: 66332b0295a18522d503f37b
paper_authors.name: Qinru Li
paper_authors._id: 66332b0295a18522d503f37c
paper_authors.name: Shangxuan Wu
paper_authors._id: 66332b0295a18522d503f37d
paper_authors.name: Han Deng
paper_authors._id: 66332b0295a18522d503f37e
paper_authors.name: Sangjin Lee
paper_authors._id: 66332b0295a18522d503f37f
paper_authors.name: Chris Sweeney
paper_authors._id: 66332b0295a18522d503f380
paper_authors.name: Qiurui He
paper_authors._id: 66332b0295a18522d503f381
paper_authors.name: Wei-Chih Hung
paper_authors._id: 66332b0295a18522d503f382
paper_authors.name: Tong He
paper_authors._id: 66332b0295a18522d503f383
paper_authors.name: Xingyi Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:47:46.474Z
paper_authors._id: 629ee11846b4826be2d94676
paper_authors.avatarUrl: /avatars/95ab04122ad626d74bf8a65d7b2bf48a.svg
paper_authors.fullname: Xingyi Zhou
paper_authors.type: user
paper_authors.user: zhouxy
paper_authors._id: 66332b0295a18522d503f384
paper_authors.name: Farshid Moussavi
paper_authors._id: 66332b0295a18522d503f385
paper_authors.name: Zijian Guo
paper_authors._id: 66332b0295a18522d503f386
paper_authors.name: Yin Zhou
paper_authors._id: 66332b0295a18522d503f387
paper_authors.name: Mingxing Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:47:16.901Z
paper_authors._id: 62b56843a1bae3c711c19a8f
paper_authors.avatarUrl: /avatars/66762980669bf6dcad6f39255f24e1f7.svg
paper_authors.fullname: mingxing tan
paper_authors.type: user
paper_authors.user: mingxing
paper_authors._id: 66332b0295a18522d503f388
paper_authors.name: Weilong Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-02T07:47:10.235Z
paper_authors._id: 63db2f729f2687298a134ec6
paper_authors.avatarUrl: /avatars/d2aee5ec8f5b674a09557d4cc3409591.svg
paper_authors.fullname: Weilong Yang
paper_authors.type: user
paper_authors.user: dragondoudou
paper_authors._id: 66332b0295a18522d503f389
paper_authors.name: Congcong Li
paper_title: STT: Stateful Tracking with Transformers for Autonomous Driving
paper_summary: Tracking objects in three-dimensional space is critical for autonomous
  driving. To ensure safety while driving, the tracker must be able to reliably
  track objects across frames and accurately estimate their states such as
  velocity and acceleration in the present. Existing works frequently focus on
  the association task while either neglecting the model performance on state
  estimation or deploying complex heuristics to predict the states. In this
  paper, we propose STT, a Stateful Tracking model built with Transformers, that
  can consistently track objects in the scenes while also predicting their states
  accurately. STT consumes rich appearance, geometry, and motion signals through
  long term history of detections and is jointly optimized for both data
  association and state estimation tasks. Since the standard tracking metrics
  like MOTA and MOTP do not capture the combined performance of the two tasks in
  the wider spectrum of object states, we extend them with new metrics called
  S-MOTA and MOTPS that address this limitation. STT achieves competitive
  real-time performance on the Waymo Open Dataset.
paper_ai_keywords: Transformers Stateful Tracking S-MOTA MOTPS MOTA MOTP long term history detections association task state estimation appearance signals geometry signals motion signals